Larian Studios Forums Divinity: Original Sin 2 - The Modding Dungeon Divinity - Original Sin 2 - Modding Tools [RELEASE] Heightmap Import Tool

Introduction
========
A tool written in C# to import grayscale heightmaps in DO:S and DO:S 2. There is also functionality for L3DT in the DO:S version but support for that has ended and there is no way to know if it still works like it did four years ago.

Current version
==========
V2.1.1

Download
=======
https://github.com/Celludriel/DOS_Terrain_Importer/files/1337059/DosTerrainImporter_v2.zip

Installation
========
Just unzip the zip file in a convenient folder and open up Dos_Terrain_Importer.exe. This program is safe and can be checked by the community on my open source github. The .net framework is needed to make the program run, although this is installed on a lot of computers by default now. Should it not work try downloading the latest version of the .net framework runtime

Usage
=====

Step 1: create a heightmap
-------------------------------
- Create a bmp file with dimensions (patchsize*33) x (patchsize*33).
- Draw your heightmap with greyscale tints where 0 is the lowest point and 255 is the highest point. I have experimented with different bit depts but saving as a 16 bit image seems to be the safest safe.

Step 2: create a terrain in divinity engine 2
--------------------------------------------------
- Create a new terrain in the level you are working on and make sure the patch size is the size you used in the formula determining the width and height of your heightmap
- Save your terrain in the engine
- Go to your divinity original sin 2 install directory and navigate to your project where your terrain is saved. fe:
\Data\Editor\Mods\Godlings_7ffdb20a-bb9e-4ca6-a3d3-46a3d0c650d3\Levels\ImportTest100x100\Terrains
- Notice the presence of a .data file with a uuid in front

Step 3: doing the import
-----------------------------
- Open up the importer
- press the DO:S 2 button
- Browse for the .data file you just created
- Browse for the bitmap heightmap file you created
- Enter the min height, this is the height the black 0 value will be at
- Enter the max height, this is the height the white 255 value will be at
- Press generate. Generating the terrain is a destructive operation for all the heightmap data, once you press it all previous terrain changes will be overwritten by the new heightmap data.

Step 4: Reload the level
----------------------------
- Go back to divinity engine 2 and choose reload level in the file menu. You should now see the imported heightmap terrain.

Sounds great, thanks a lot

Thank you! I'm grabbing it now and will post any results from it. I would highly recommend adding this to the Community Page Submission section of the wiki if you have the time!

Also, might be good to add a README file in your zipped archive that has the notes you posted here on how to use.

experiment with it first, maybe you'll find some bugs that I need to fix. I'm still puzzled why I can't make perfect vertical walls though. There must be some explanation for it maybe because it all works with triangles I don't know yet.



Will do in a bit when I'm a bit less busy

Good job Celludriel!



I have a lot of patch files created in my terrain folder. In this case I tried selecting the first one but it doesn't work.

Tested with a 8x8 terrain.

Can you get me the terrain patch and data file together with your heightmap ? I'll try to reproduce why it wouldn't work for you.

Also make sure you are in the \Data\Editor\Mods directory. I made the mistake not being in the Editor folder the first time and it didn't seem to work either then.

Also each terrain gets an unique UUID, that long weird text thing. You should be able to find that in the divinity editor as well if you click on the properties of your terrain. Make sure you pick that file.

Yes, the path is correct but I have lots of patch files (64 files for a 8x8 terrain).

The editor crashed when I click "reload level", so the files were modified but something is broken. Maybe 8x8 is too big? or my bmp has something wrong.

I sent you a pm with files.

oh I see, you are right if you go bigger there are multiple patch files ... damn it all. This makes it way more complex then anticipated.

Once again Larian violated the KISS principle. I have to rethink everything now ... I'm guessing each 1x1 is a patch file. Not sure yet I have to think about this.

Yes, it seems like the terrain composed of these patchs (like plots) and generate one patch file for each of these segments.

Well, we are here to help and test things just patience now.

I just did a test, by using my same heightmap for a 1x1 patch for all the patch files



So it is as I feared and you said so as well they cut up the terrain for each patch. This complicates things quite a bit. That means I need to cut up the heightmap on parsing and then pick which patch files to update and hope I pick the right order as well.

I also have to determine how many patch files there are based on the size of the bitmap. This is a major rewrite from the code from something simple to something more elaborate

Keep faith, and thanks for your hard work already.

Hi, i just dont get why would i use it at all? Sorry for that lame question but i dont get it :-)

A lot has to do with cutting your workload in half. By drafting an idea you have for a terrain first on a simple 2D greyscale image, using some other tools you can get some very good results. One of the things DO:S E suffers vs NWN is that it takes way to long to create new content due to some of the clunkyness of the UI. That's how I feel at least.

They offer a GM mode in DO:S 2 which is a great great feature. But if a GM needs three months to prepare a 4 hour DM session it's also a failure. I'm trying to find out tools that will cut a GM's workload drastically so this time the DO:S 2 content generating doesn't stop to a grinding halt after just a few weeks ... like last time ... So I have a clear goal in mind with all I'm doing here

For more information about heightmap manipulation.

https://en.wikipedia.org/wiki/Heightmap

Great spirit and motivation you show :) thx for headsup

I think I fixed it. I made an attempt with a 2x2 patch which translated to a 66px x 66px bitmap.

Heightmap:
[img:center]http://users.telenet.be/sunspot/dos/patch66x66.bmp[/img]

Result:



more testing is probably needed and I have to make a new release, but this is looking well.

UPDATE
=====
OP updated with the new release and usage updated as well. Take note you now have to use the DATA file no longer the patch file.

Currenlty only use square terrains, there is still a problem with 2x1 or 1x2. It has something to do with how X and Y are mirrored and reversed in the patch file vs the bitmap file and it's giving me a headache.

Wow, that is amazing: In 3 seconds i made a volcano. (4x4 patch) Nothing nice so far but really fast and much better base than fiddling 1 hour with terrain shaping tools:


Is there a good height map data bank? Where you find stuff like Minas Tirith or whatsoever..?

EDIT:

This island setup was also super fast and smooth. Seems like BMP really has to be 16bit (mustnt 24bit)

Nice work, seems it is working as expected so far. I also fixed that you can't use none square setups. I just need to make a release and make a build for it.

First I need food and a movie though. Have been working on this since I woke up this morning. I'll make sure to release it later today. For now you can experiment with the current version

Glad it is paying of for you !

Awesome tool, thank you for that :)

Will this override existing terrain paints and layers? I'd like to improve the heightmap and apply the changes to the terrain agian. If this works, exporting an existing terrain to bitmap to change and reimport it would be awesome, too.

It only overwrites the heightmap data in the patch files, I suggest you make a backup of your work, and then try it. I think it should work

Just made some minas tirith like base myself in photoshop:

Nice one, seems you at least got a vertical wall going right, didn't work with my tests.

I have updated the main post with a new version the problem with non squared terrains is hopefully fixed I tested it with a 2x1 and included an example heightmap for it

I wont work with this because I'm lacking time resources to really fill that kind of huge level right now. Just wanted to showcase the potential of your awesome tool. The verical walls aren't beautiful, as place itrms always eill look better,but using heightmaps is a huge help to generally layout a level. It's really great to have that tool. Still i beg larian to provide some API to have something like this as a plugin.

Keep on working on this. I just hope to have encouraged you to continue polishing this nice one.

I would love designing this as a plugin, as for polishing , not much more I can do the basic functionality is in. The only thing I can think of is autotexturing by use of a climate map. But that means having to relearn how those damn pages in the Larian file format work. I remember from last time this is a very complicated material to master. Not sure I'm up for that right now, I'm actually with vacation and shouldn't be coding ....

Here my tests and my thoughts;

My main concern, the resolution for each patch is really low, the smooth is needed if you want avoid some ugly hard edges in heigh levels. This can be fixed using higher resolution heightmaps, so we can have more pixels used per square (patch). My doubt here; are we forced in dos2 engine to use this formula (33 pixels per patch)?

Can we go higher? this is a must have if we really want to have high detailed terrains. Maybe Larian should take a look on this, they could make this even better if they put some effort.

Here the example of these issues;

- The need of smooth tool if you want avoid shadow issues due to hard edges.



- Here we can see how this low res affecting painting too.



- A smoothed terrain, my best approach.

Rada, you may have been using a base terrain texture in that second image leading to the chunky look.

Ok, I'll check that.

I would need a confirmation of a Larian employee but I'm afraid we are stuck with it. In one patch file the heightmap is 4356 bytes. Each height point in the heightmap is 4 bytes depicting the height in meters. Taking this in account we need to figure out how much space there is in one patch.

4356 / 4 = 1089

So we have 1089 height points in total that can be present in the heightmap, considering we have a square patch we need to take the square root to have one width.

SQRT(1089) = 33

So we end up with a heightmap of 33 * 33 ... if there is any other way to increase resolution I'm more then willing to find out. I'm afraid there isn't though

Need to play with it a bit more but it will definitely come in handy. It may be coincidence, but it didn't seem to work for me when I tried setting the max height from 20 to 10? It didn't re-generate the terrain at all a couple of times when I tried using the exact same paths and not sure why.

Great work, this will help a lot.

Can you send me the heightmap and the dimension you were using. I can then experiment a bit with the min and max height. Weird though that is one part I haven't had problems with yet. I'll try and figure out what is going on there.

@moderator You can move this thread to the tools forum if you want. It kind of belongs there

I tried to use this and it doesn't seem to be working. Is this still functional?

Same problem. I've tried a bunch of combinations using his example heightmaps and none work. I also tried some earlier revisions and they do not either. I wonder if something changed in the terrain data format.

I'll take a look tomorrow, hopefully they didn't change to much to the file format, but if they did I need to figure out what they did

UPDATE:

Are you sure it's not working ? I'm using the steam edition of the editor so it should be updated to the latest version. I tried the version of the zip file, tried the version in my current workspace of visual studio, tried different test files. It all still seems to work ?

Can you please verify you are using the up to date editor from steam if you are send me your terrain files and heightmap file please so I can investigate what is going wrong with your files.

Maybe try again with a new level. For me it wasnt working several times. Always had to retry from scratch with new level i not made any attempts so far. Dont know why it just worked sometimes. Maybe Celludriel you could have look into this.

Well I just made a new project, new level and a new terrain. It just worked as expected. Maybe it's a dimension issue where my math is wrong figuring out the x and y coordinates ?

Sent you a youtube video recording my workflow in case that helps. I did do a new level when testing. Thanks for your time.

The version in your video is a bit fuzzy, so I couldn't make it out perfectly. Your workflow is right though. I replicated your steps and here it works like it should. I'm stomped atm why it wouldn't be working for you.

This is my version of the editor



I did notice the timestamp on your patch files didn't change at all. As if that there was no write operation to them at all. On thing I'm might be thinking is that there isn't any write access to the folder. It's a longshot though, since you where able to do it before.

I was wondering the same thing, so I tried all kinds of things tonight with running as administrator, moving the tool to be on the same drive, opening new projects, opening old projects, trying with new terrains, and trying with old terrains. It worked once out of about 8 permutations and I have no idea why that one worked while all the others fail.

All my tries keep working consistently, I just can't reproduce it. Keep in mind that the tool itself isn't that complicated. It's barely 50 lines of code. It just reads two files and rewrites one of them.

All the code is in two files

https://github.com/Celludriel/DOS_Terrain_Importer/tree/master/DosTerrainImporter/Importer/Dos2

Dos2GrayScaleImporter.cs: figures out the name of the patch files and order
PatchFileWriter.cs: reads and writes one patch file

Wish I could reproduce your problem, but I get consistent results

Thanks for your dedication to the tool, Celludriel.

I seem to have a workflow that gets consistent success now. I don't know which of these exactly resolves my former issues, but I was able to make a good 6-7 maps in a row this morning. Things I am doing:

1. Create a new level with only one terrain and save.

2. Always choose a normal 'exit' from the editor. I was in the habit of choosing 'force exit' from the extended period of time when the editor would hang and crash from a normal exit but that seems to be mostly resolved at this point.

3. Run the heightmap tool in administrator mode.

4. Fill-in the bitmap file first and editor terrain second.

My guess is that #2 in the list is the real difference maker. You can see some built-in source control stuff going on when saving a level and perhaps there is some kind of internal locking happening that does not get released when you force exit from the editor.

I do still get crashes when attempting to import heightmaps with a level size of 10x10, but since that is too big for my needs anyway, not a problem for me.

Thanks again, this tool is a huge help to me.

It took me a while to figure this out but it seems to work for me. Nice tool! I'll be using it.

How trivial would it be to make it work the other way around? I'm trying to get the DOS:1 maps into DOS:2. There's probably an easier way to do that, but I wouldn't know where to start.