TresCom Forums

TresCom : The Trespasser Fan Gateway...
It is currently Sat Sep 21, 2019 3:24 am

All times are UTC




Post new topic Reply to topic  [ 64 posts ]  Go to page 1, 2, 3, 4, 5  Next
Author Message
PostPosted: Sun Jun 30, 2019 10:00 pm 
Offline
Dinosaur egg
Dinosaur egg
User avatar

Joined: Wed Jun 06, 2018 8:02 pm
Posts: 22
I've been playing around with a few image up-scalers that use machine learning to guess what the enlarged image would look like rather than just average out the pixels.
I though it'd be fun to run the trespasser textures through it.
Doubling them seemed to be a good compromise between size and benefit. Unfortunately the original images are only 8 bit colour so there are colour banding issues. They might benefit from some blurring?

Current Version
V2.0 http://www.trescom.org/download/downloads/misc/NNHD_Trespasser_Textures_V2.0.7z


Attachments:
PPrecisionSMLcrate_t2.jpg
PPrecisionSMLcrate_t2.jpg [ 47.96 KiB | Viewed 2590 times ]
Ppaintcan04t2.jpg
Ppaintcan04t2.jpg [ 55.23 KiB | Viewed 2590 times ]
Pfuelbarrel03t22.jpg
Pfuelbarrel03t22.jpg [ 41.33 KiB | Viewed 2590 times ]
PBrCanNoTopHi_lites00t2.jpg
PBrCanNoTopHi_lites00t2.jpg [ 58.25 KiB | Viewed 2590 times ]


Last edited by Isolt on Wed Aug 21, 2019 4:40 am, edited 3 times in total.
Top
 Profile  
 
PostPosted: Mon Jul 01, 2019 4:21 pm 
Offline
T-Rex Killer
T-Rex Killer
User avatar

Joined: Thu Apr 24, 2003 11:20 pm
Posts: 11510
Location: Lima, Peru
Nice results...

_________________
Visit The Carnivores Saga - a forum devoted to modding Action Forms' Carnivores, Carnivores 2 and Carnivores: Ice Age games
Tres WIP: updated T-Script Reference and File Formats documents
Sound name listings for the Demo (build 117), Retail (build 116), Beta 103, Beta 99, Beta 97, Beta 96, Build 55, PC Gamer Alpha (build 32) and E3 1998 Alpha (build 22) TPA files


Top
 Profile  
 
PostPosted: Mon Jul 01, 2019 8:06 pm 
Offline
Dinosaur egg
Dinosaur egg
User avatar

Joined: Wed Jun 06, 2018 8:02 pm
Posts: 22
Thanks.
They'll never be as clear as TheIdiot's remade ones, but they're better than any standard up-scaling.
It's not hugely noticeable in game but things are a bit more crisp.
Now to sort the texture overlaps from every level... There must be an easier way.

Image
Image
Image
Image
Spoiler: show
Attachment:
TPass005.jpg
TPass005.jpg [ 609.23 KiB | Viewed 2572 times ]

Spoiler: show
Attachment:
TPass004.jpg
TPass004.jpg [ 485.62 KiB | Viewed 2572 times ]

Spoiler: show
Attachment:
TPass002.jpg
TPass002.jpg [ 420.75 KiB | Viewed 2572 times ]

Spoiler: show
Attachment:
TPass001.jpg
TPass001.jpg [ 512.77 KiB | Viewed 2572 times ]


Top
 Profile  
 
PostPosted: Tue Jul 02, 2019 4:22 am 
Offline
T-Rex Killer
T-Rex Killer
User avatar

Joined: Thu Apr 24, 2003 11:20 pm
Posts: 11510
Location: Lima, Peru
How many textures did you apply this process to?
And yes, the textures having been reduced to 8-bit palettes in the final step of the export for the game makes them somewhat problematic to improve... does the process get any advantage if you convert them to 24-bit RGB images first? At leas, normal resampling algorithms usually work better if you do that and then convert back to an 8-bit palette... and with CE, you don't really need to make them 8-bit again.

_________________
Visit The Carnivores Saga - a forum devoted to modding Action Forms' Carnivores, Carnivores 2 and Carnivores: Ice Age games
Tres WIP: updated T-Script Reference and File Formats documents
Sound name listings for the Demo (build 117), Retail (build 116), Beta 103, Beta 99, Beta 97, Beta 96, Build 55, PC Gamer Alpha (build 32) and E3 1998 Alpha (build 22) TPA files


Top
 Profile  
 
PostPosted: Tue Jul 02, 2019 5:22 am 
Offline
T-Rex
T-Rex
User avatar

Joined: Thu Feb 20, 2014 9:13 pm
Posts: 2853
Location: Canada, eh?
I was waiting for someone to do this. Had wanted to do it myself a while back but couldn't find a (free) tool to do so nicely, and retexturing everything from scratch takes a lot of time, especially considering a lot of Tres' objects have a large number of split-up textures, such as dinosaurs and vehicles.

The colour banding in the original images does suck, but there isn't a whole lot you can do about that without remaking the textures from scratch. You could try blurring the image a bit before upscaling but you'd lose some of the details. With CE you can use hardware textures in .dds format which should allow you to avoid having to convert the images over to 256 colors - I found saving them as .bmps causes some issues with lighting.

What I'd like to see now is a conversion of all of the original Trespasser bumpmaps to CE-compatible normalmaps so that they work in fan-made levels, since GeomAdd never got the ability to use standard Tres bumpmaps in custom levels.

_________________
Image
Return to Jurassic Park: A Trespasser Mod - Forum or ModDB | Trespasser HD | My Release Thread | Level Reviews


Top
 Profile  
 
PostPosted: Tue Jul 02, 2019 7:43 am 
Offline
T-Rex Killer
T-Rex Killer
User avatar

Joined: Thu Apr 24, 2003 11:20 pm
Posts: 11510
Location: Lima, Peru
TheIdiot wrote:
What I'd like to see now is a conversion of all of the original Trespasser bumpmaps to CE-compatible normalmaps so that they work in fan-made levels, since GeomAdd never got the ability to use standard Tres bumpmaps in custom levels.

Technically, it shouldn't be too hard. Especially since, technically (yes, again), Trespasser isn't using bumpmaps but rather normal maps, only that instead of using cartesian coordinates to store the vector normals (X, Y, Z), it's using spherical coordinates (radius, azimuth and elevation).

Since by definition a normal vector always has a length of 1.0 (it's called a normal because it has been normalized), its radius in spherical coordinates will always be 0, so it doesn't need to be stored; the azimuth (horizontal angle) will go from 0 to 360 degrees (think of compass directions, with 0 being north, 90 being east, 180 being south, 270 being west and 360 back to north again), and while the elevation should go from -90 (pointing down) to +90 degrees (pointing up), the normal vector of a surface you're looking at will never have a negative angle, so it only has to go from 0 to 90 degrees. That's the reason why, from the 10 bits used for storing the normal maps (the other 6 bits being the palette index for a 64-color image), 6 bits are used for the azimuth and only 4 bits for the elevation. 6 bits mean 64 possible values, while 4 bits mean 16 possible ones, which is 4 times less, just kile 90 degrees is 1/4th of 360 degrees. The resolution is the same in both cases, 360/64 = 90/16 = 45/8 = 5.625 degrees.

Converting Trespasser's normal maps into bump maps has been more difficult than it would be converting them to regular normal maps, which only implies converting the spherical coordinates into cartesian ones and can be done by simple multiplication. Converting to bumpmaps, OTOH, has required numerical integration, and the results aren't accurate.

SWPext (don't remember if TresEd does too) has an option to extract the bumpmapped textures as raw data; then all we need to do is have another program process those files to write the corresponding standard normal maps as BMP or PNG images.

_________________
Visit The Carnivores Saga - a forum devoted to modding Action Forms' Carnivores, Carnivores 2 and Carnivores: Ice Age games
Tres WIP: updated T-Script Reference and File Formats documents
Sound name listings for the Demo (build 117), Retail (build 116), Beta 103, Beta 99, Beta 97, Beta 96, Build 55, PC Gamer Alpha (build 32) and E3 1998 Alpha (build 22) TPA files


Top
 Profile  
 
PostPosted: Tue Jul 02, 2019 12:05 pm 
Offline
-=TresCom Website Manager=-
-=TresCom Website Manager=-
User avatar

Joined: Fri Jun 24, 2005 9:40 pm
Posts: 3761
Location: Sweden
A member on the Discord have done this to several LAB textures and upscaling some quite good.

IIRC the SWPext have issues and export the bumpmaps with some lines in them? However, you can use the GUIApp from CE and export textures along with the bumpmaps. :)

_________________
"...there used to be more benches, but InGen's workers removed them during the evacuation in the name of framerate."

"The main laboratory and administrative buildings. This is where we made our work, where the real magic trick happen. When they are in need of height fixing, they'll come here." - Hammond


Top
 Profile  
 
PostPosted: Tue Jul 02, 2019 7:48 pm 
Offline
T-Rex Killer
T-Rex Killer
User avatar

Joined: Mon Dec 06, 2004 5:21 pm
Posts: 11740
Location: Portland, OR
Hey now, those upscaled images are great! I was unaware of this technique, am impressed...

Also.. spherical coordinate systems. Nice, I wonder if there are alien races who use this as their standard model rather than X,Y,Z box-coordinates..

_________________
Image
Image Image T-Script Reference / Trespasser Millennium / JP Dark Secrets / Tres File Box


Top
 Profile  
 
PostPosted: Tue Jul 02, 2019 8:52 pm 
Offline
Dinosaur egg
Dinosaur egg
User avatar

Joined: Wed Jun 06, 2018 8:02 pm
Posts: 22
Hi machf.
I've done all of beach textures so that I could test it.
Last night I combined the textures from all the levels to avoid double ups. There was a note in the GUI app about having different textures with the same name in different levels. I went through them as I combined them but didn't notice any. Does anyone know of any exceptions? There are thousands of files and I may have overlooked some.
For my beach test I didn't upscale the normal maps, or even use the ones extracted along with the textures. Would those benefit from up-scaling?
I know ground textures aren't applied in CE so I won't do these. How do I identify them? "TGrass05t2" looks like a good candidate, what about things like "TBchClff30t2" and "TgiantBoulder01t2"
Would imposter sprites would benefit from up scaling, or are the new draw distances so good it's not necessary?

The Idiot, I want to avoid modifying the textures before up-scaling them. The less they've been modified the better the results are I believe. Hmm lighting issues with .BMP is concerning as that's what I was planning to use. TGA is an option, or I could batch convert them to .DDS with something?

My current plan is exporting all textures with GUIApp from CE. Combining them to remove doubles. Removing ground textures and normal maps. Upscale 2x with 24bit BMP's as the output. Then possibly add a bit of selective blurring on some of the worst banding.

Thanks tatu, you helped me get the GUIApp running over in the CE thread so that I could export these.


Top
 Profile  
 
PostPosted: Tue Jul 02, 2019 9:36 pm 
Offline
T-Rex Killer
T-Rex Killer
User avatar

Joined: Thu Apr 24, 2003 11:20 pm
Posts: 11510
Location: Lima, Peru
tatu wrote:
IIRC the SWPext have issues and export the bumpmaps with some lines in them? However, you can use the GUIApp from CE and export textures along with the bumpmaps. :)

The idea is not to export them as bumpmaps (that's why in the process they get those lines), but the raw data, and then convert THAT into a regular normal map...

Isolt wrote:
Hi machf.
I've done all of beach textures so that I could test it.
Last night I combined the textures from all the levels to avoid double ups. There was a note in the GUI app about having different textures with the same name in different levels. I went through them as I combined them but didn't notice any. Does anyone know of any exceptions? There are thousands of files and I may have overlooked some.

Nice amount of work that must have been...
The thing is, different levels use different palettes, for optimization reasons. So, what was originally the same 24-bit RGB texture in the source 3DS MAX files, may have been converted to 8-bit using a certain palette when exporting it for level A, and a different palette when exporting it for level B. And since the downconversion from 16 million colors to 256 colors just uses the "nearest color match" approach, they end up looking different.
Then there's the issue that, internally, Trespasser stores the texture names as CRC-32 hashes, and from a certain version onwards (at least Build 55), the hash is applied not just to the name itself but to the float Diffuse value appended to it... and when different objects with different Diffuse values use the same texture, it's actually stored twice for that reason, even if it's identical.

Quote:
For my beach test I didn't upscale the normal maps, or even use the ones extracted along with the textures. Would those benefit from up-scaling?

Maybe, but let's wait until after we've generated the standard normal maps...
And I'm not really sure the same process would apply, since normal maps, even though they are stored as 24-bit RGB pictures, aren't really color pictures, just data that is stored as color pictures for convenience (X, Y and Z stored as the R, G and B values, I think, or maybe B, G and R).
Quote:
I know ground textures aren't applied in CE so I won't do these. How do I identify them?

They usually have names that begin with "Trn", "TrnObj", or sometimes, "T" (or both: "TTrnObj")
Quote:
"TGrass05t2" looks like a good candidate, what about things like "TBchClff30t2" and "TgiantBoulder01t2"

Unfortunately, the Giant Boulder isn't a terrain feature, but a regular object...
Quote:
Would imposter sprites would benefit from up scaling, or are the new draw distances so good it's not necessary?

Imposter sprites are generated on the fly; I think you may mean the LoD meshes. And they sometimes use the same textures as the regular meshes.

_________________
Visit The Carnivores Saga - a forum devoted to modding Action Forms' Carnivores, Carnivores 2 and Carnivores: Ice Age games
Tres WIP: updated T-Script Reference and File Formats documents
Sound name listings for the Demo (build 117), Retail (build 116), Beta 103, Beta 99, Beta 97, Beta 96, Build 55, PC Gamer Alpha (build 32) and E3 1998 Alpha (build 22) TPA files


Top
 Profile  
 
PostPosted: Tue Jul 02, 2019 10:15 pm 
Offline
-=TresCom Developer=-
-=TresCom Developer=-
User avatar

Joined: Sun Nov 10, 2002 10:26 pm
Posts: 5388
Location: That country nobody likes (you know the one)
Quote:
SWPext (don't remember if TresEd does too) has an option to extract the bumpmapped textures as raw data; then all we need to do is have another program process those files to write the corresponding standard normal maps as BMP or PNG images.


Presumably, Andres' bumpconvert utility does that, machf. I never used it, but if he released it, I'd assume that it does indeed work.


Top
 Profile  
 
PostPosted: Tue Jul 02, 2019 10:55 pm 
Offline
Dinosaur egg
Dinosaur egg
User avatar

Joined: Wed Jun 06, 2018 8:02 pm
Posts: 22
Quote:
The thing is, different levels use different palettes, for optimization reasons. So, what was originally the same 24-bit RGB texture in the source 3DS MAX files, may have been converted to 8-bit using a certain palette when exporting it for level A, and a different palette when exporting it for level B. And since the downconversion from 16 million colors to 256 colors just uses the "nearest color match" approach, they end up looking different.
Then there's the issue that, internally, Trespasser stores the texture names as CRC-32 hashes, and from a certain version onwards (at least Build 55), the hash is applied not just to the name itself but to the float Diffuse value appended to it... and when different objects with different Diffuse values use the same texture, it's actually stored twice for that reason, even if it's identical.

So slight colour variations but the same file names? Do you have any examples? I started off looking carefully at each double up but quickly gave up.
I could just keep each texture pack in it's own level folder and just live with double ups. The first level is about 250MB @ 2x with no normal maps, about 1GB at 4x.

Quote:
Maybe, but let's wait until after we've generated the standard normal maps...
And I'm not really sure the same process would apply, since normal maps, even though they are stored as 24-bit RGB pictures, aren't really color pictures, just data that is stored as color pictures for convenience (X, Y and Z stored as the R, G and B values, I think, or maybe B, G and R).

Hmm no idea how they'd respond to upscale then, should make the download smaller without them. I will have to manually remove them though *sigh*

Quote:
Unfortunately, the Giant Boulder isn't a terrain feature, but a regular object...

I suspected, so much for consistency. I'll remove the obvious terrain ones and leave any I'm unsure about, shouldn't be a large overhead in filesize.

Quote:
Imposter sprites are generated on the fly; I think you may mean the LoD meshes. And they sometimes use the same textures as the regular meshes.

Ah I thought all the tiny tree textures were them

Quote:
Hey now, those upscaled images are great! I was unaware of this technique, am impressed...

Drac I did cherry pick those :P I'm still experimenting, I made this beautiful watercolour barrel painting
Spoiler: show
Attachment:
PblFuelBarrelF00t2.bmp
PblFuelBarrelF00t2.bmp [ 1.5 MiB | Viewed 2547 times ]


Top
 Profile  
 
PostPosted: Tue Jul 02, 2019 11:21 pm 
Offline
T-Rex Killer
T-Rex Killer
User avatar

Joined: Thu Apr 24, 2003 11:20 pm
Posts: 11510
Location: Lima, Peru
Rebel wrote:
Quote:
SWPext (don't remember if TresEd does too) has an option to extract the bumpmapped textures as raw data; then all we need to do is have another program process those files to write the corresponding standard normal maps as BMP or PNG images.


Presumably, Andres' bumpconvert utility does that, machf. I never used it, but if he released it, I'd assume that it does indeed work.

Nah, it converts Trespasser spherical normal maps into bump maps. What we need is another util to convert spherical normal maps into cartesian normal maps.

Stealing Andres' pictures from his original article to clear it up:

ImageImage
This is a representation of the 16-bit "bumpmapped" images in Trespasser after splitting the highest 6 bits a a paletted image map and the lower 10 bits as a curved normal map.

Then, if we split the 10 bits of the curved normal map into 6 bits for the azimuth and the lowest 4 bits for the elevation, it becomes:
ImageImageImage

Andres also showed what the X and Y components of the normal would look like after converting from spherical normals to cartesian normals:

ImageImage

He didn't calculate the Z component, but it's as easy as X and Y, and it would look similar too. Then we would just combine all three (8 bits each) into a 24-bit RGB image.
Here I've just averaged the X and Y images to create a fictional "Z" image (no, that's NOT the actual way in which it's calculated) just to show the results:
Attachment:
NormalImage9.jpg
NormalImage9.jpg [ 5.01 KiB | Viewed 2547 times ]


And after combining all three greyscale images into a single RGB one, we get:
Attachment:
NormalImage18.jpg
NormalImage18.jpg [ 6.09 KiB | Viewed 2547 times ]


That's what our standard cartesian normal map would look like, more or less. In some places it's grey, in others it veers towards cyan ("anti-red", so to speak) or towards magenta ("anti-green")...


Or just using the elevation part as an approximation of the Z component:
Attachment:
NormalImage19.jpg
NormalImage19.jpg [ 4.79 KiB | Viewed 2544 times ]


We then get this as the resulting normal map after combining all three:
Attachment:
NormalImage20.jpg
NormalImage20.jpg [ 6.35 KiB | Viewed 2544 times ]


Since here the Z component doesn't look too similar to the X and Y components, instead of a mostly grey image we get a mostly blue-violet one... if you look for normal maps online, you'll see that they tend to look like this.


Andres didn't perform this step, instead to generate bump maps, he performed numerical integration on both the X and Y components, with the results we already know, diagonal artifacts and all... but we don't need bump maps anymore for CE, we just need normal maps, and those don't need any complicated math that will end up with any artifacts.

_________________
Visit The Carnivores Saga - a forum devoted to modding Action Forms' Carnivores, Carnivores 2 and Carnivores: Ice Age games
Tres WIP: updated T-Script Reference and File Formats documents
Sound name listings for the Demo (build 117), Retail (build 116), Beta 103, Beta 99, Beta 97, Beta 96, Build 55, PC Gamer Alpha (build 32) and E3 1998 Alpha (build 22) TPA files


Top
 Profile  
 
PostPosted: Tue Jul 02, 2019 11:34 pm 
Offline
T-Rex Killer
T-Rex Killer
User avatar

Joined: Thu Apr 24, 2003 11:20 pm
Posts: 11510
Location: Lima, Peru
Isolt wrote:
Ah I thought all the tiny tree textures were them

Ah, you mean the mipmaps... they are stored in order to save time, instead of having the came recalculate the texture at further distances (when it's shrunk), they're precalculated and stored in the .PID/.SWP files. I don't know how CE handles them It's likely that if you use .DDS textures, the mipmaps will be contained within them, too.

_________________
Visit The Carnivores Saga - a forum devoted to modding Action Forms' Carnivores, Carnivores 2 and Carnivores: Ice Age games
Tres WIP: updated T-Script Reference and File Formats documents
Sound name listings for the Demo (build 117), Retail (build 116), Beta 103, Beta 99, Beta 97, Beta 96, Build 55, PC Gamer Alpha (build 32) and E3 1998 Alpha (build 22) TPA files


Top
 Profile  
 
PostPosted: Wed Jul 03, 2019 1:59 am 
Offline
T-Rex
T-Rex
User avatar

Joined: Thu Feb 20, 2014 9:13 pm
Posts: 2853
Location: Canada, eh?
machf wrote:
Isolt wrote:
Ah I thought all the tiny tree textures were them

Ah, you mean the mipmaps... they are stored in order to save time, instead of having the came recalculate the texture at further distances (when it's shrunk), they're precalculated and stored in the .PID/.SWP files. I don't know how CE handles them It's likely that if you use .DDS textures, the mipmaps will be contained within them, too.

So long as you generate mipmaps when you export your .dds files, yes, it does appear to use the mipmaps from the .dds files. The impostors however do not appear to benefit from HD textures and instead use the normal textures which are contained in the .swp instead.

_________________
Image
Return to Jurassic Park: A Trespasser Mod - Forum or ModDB | Trespasser HD | My Release Thread | Level Reviews


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 64 posts ]  Go to page 1, 2, 3, 4, 5  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 9 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron

Powered by phpBB® Forum Software © phpBB Group