Development Blah

All other topics around Descent and D2X-XL

Moderators: Frustikus, simX, Aus-RED-5

User avatar
karx11erx
D2X-XL Master
D2X-XL Master
Posts: 8112
Joined: Mon Sep 24, 2007 8:48 pm
Location: Wilferdingen, Germany
Contact:

Re: Development Blah

Postby karx11erx » Wed Jun 11, 2014 11:12 am

Mint is up and running. Last thing to do is to setup the boot loader to be able to chain to my Windows 7 installation.
User avatar
karx11erx
D2X-XL Master
D2X-XL Master
Posts: 8112
Joined: Mon Sep 24, 2007 8:48 pm
Location: Wilferdingen, Germany
Contact:

Sound Issues

Postby karx11erx » Wed Jun 11, 2014 11:54 am

Hires sound usage will now be turned off temporarily by d2x-xl if there hasn't been loaded a hires sound for every default (built-in) low res sound. Hires sounds can come both from the basic d2x-xl installation and from mods. This implies that a mod can enable hires sound if providing all required hires sound files even if the default hires sounds have not been installed. This is done on a per level base, as mod can provide sound files both globally for the entire mission and per level.

This implies that no add-on sounds will be played back if hires sound usage gets disabled, even if present in a level, since these are all sampled at high resolution.

This means that D2X-XL does not require the hires sounds to be installed. If it doesn't find them, it will configure the sound system so that the standard Descent sounds are correctly played (accepting a loss of quality compared to the hires sounds).

The underlying problem is that D2X-XL's previous method of resampling (22 KHz 8 bit mono) low res sounds no hires sound could be found for to 44 KHz 16 bit stereo doesn't go well with SDL_mixer (sound dies when many sound sources are present).
User avatar
karx11erx
D2X-XL Master
D2X-XL Master
Posts: 8112
Joined: Mon Sep 24, 2007 8:48 pm
Location: Wilferdingen, Germany
Contact:

Re: Development Blah

Postby karx11erx » Thu Jun 12, 2014 12:24 am

I almost broke my Mint installation when messing around with the boot loader :sweat: ... I had to fix it using the live Mint CD and some volume mounting and chroot magic. :coding:

Everything is working fine on Mint though, particularly Subclipse. Just the usual minor problems, like the Eclipse IDE I had installed via the package management not finding JavaHL despite an up to date version being on my system and my properly adding the folder to what I believed was eclipse's ini file, requiring me to manually install it (in my home folder ...) and configure it there. Oh well, peanuts.
User avatar
karx11erx
D2X-XL Master
D2X-XL Master
Posts: 8112
Joined: Mon Sep 24, 2007 8:48 pm
Location: Wilferdingen, Germany
Contact:

Re: Development Blah

Postby karx11erx » Fri Jun 13, 2014 5:16 pm

After having Mint running to my satisfaction, I finally found some time to work on D2X-XL's Linux code, and I promptly ran into a couple of bugs, one of which could cause problems in the 64 bit Windows version of D2X-XL, too, and this one (which had to do with reading compressed mesh data) was a bit tricky to figure. Fortunately, I managed to determine what caused it and could fix it.

Today I was back at the Windows version, working over the folder management again ... I have to say that while the ideas about how to properly structure D2X-XL's data and folders steel's initial proposition had inspired make a lot of sense, they also cause a lot of problems. You need to be very careful not to create or use a wrong folder somewhere. Allowing for user-specified game, shared and private data folders makes things even more complicated. Mod handling adds yet another stage of complexity. As soon as a mod is available for the mission to be loaded, D2X-XL has to check up to three different locations for its data: The level specific mod folder, the global mod folder and the general data folder. I think I have finally got this working though.
User avatar
karx11erx
D2X-XL Master
D2X-XL Master
Posts: 8112
Joined: Mon Sep 24, 2007 8:48 pm
Location: Wilferdingen, Germany
Contact:

I told ya ...

Postby karx11erx » Sat Jun 14, 2014 1:48 pm

that I am always finding bugs and flaws in d2x-xl nobody else ever seems to notice ... :strange:

Today, I was playing around with mesh improvement. In case you don't know what that is: It splits the faces of the level walls into smaller faces, resulting in improved lighting (at the potential cost of a reduced frame rate, depending on your computer's power). There is a nice article in my worklog about it.

Well, when loading my multiplayer level Subspace (which has a very cool subterranean cave and cave lake theme ;)), I found that its lake had dark spots, or rather bright spots surrounded by dark areas. Now this level's lighting has a peculiarity: Since there are no "natural" light sources in it, I had assigned a low brightness value to many of its faces including the lake, effectively turning these faces into light sources. Now d2x-xl gives the full brightness values assigned to such a face to each of the face's vertices ("corners"). In case to do so, the lighting procedure has to determine whether a vertex belongs to a given light source potentially illuminating it. However, a level's segments do not have any information about the extra vertices created during mesh improvement, so that test failed for such vertices if they were part of a light emitting face.

I had never gotten aware of that before, because I hadn't been using mesh improvement for years (I was rather relying on lightmaps), and because you only see that for certain faces of a level. Fortunately, it only took me about an hour to figure this issue. Unfortunately, it seems to be harder to fix it than to figure it ...
User avatar
karx11erx
D2X-XL Master
D2X-XL Master
Posts: 8112
Joined: Mon Sep 24, 2007 8:48 pm
Location: Wilferdingen, Germany
Contact:

Bugs nobody seems to notice ...

Postby karx11erx » Thu Jun 19, 2014 3:24 pm

... sometimes people find bugs even I fail to detect. The player who probably has the keenest eyes for these is ... Pumo! I am asking myself whether he is looking for them with a microscope at times. :P

Pun aside, his gift of observation is precise and very helpful. Part of that may have to do with the fact that he is one of the most prolific and best Descent level designers out there, not being satisfied with anything less than perfection and always pushing the envelope with his D2X-XL mods. So it is no wonder that he stumbles even over minor issues in the program.

To make a long story short: There is a new D2X-XL version available. You hadn't guessed that, had you? ;)
User avatar
karx11erx
D2X-XL Master
D2X-XL Master
Posts: 8112
Joined: Mon Sep 24, 2007 8:48 pm
Location: Wilferdingen, Germany
Contact:

"Fast" glow rendering

Postby karx11erx » Fri Jun 20, 2014 9:56 am

The "fast" option renders significantly smaller screen areas around the glow effect, while the "full" option always renders the entire screen. For the "fast" option, D2X-XL therefore tries to determine the position and size of the screen area where the glow effect is. Let's take a lightning bolt. When rendering it, d2x-xl computes the screen positions of its vertices (corner points) to determine a rectangular screen area containing that lightning bolt. When rendering glow for that lightning bolt, it subsequently only renders to that screen area (which it makes a little bit bigger before because glow makes the effect bigger). For small and/or distant effects, this will of course result in significantly less data to be moved around on the graphics card.

What glow does is to take an image and soften it using a Gaussian blur. The blur offset I am using is *two* pixels. At low screen resolutions, this may result in "ghosting". However, I went down as far as 800x600 when testing this and couldn't see any ghosting on my computer.
User avatar
karx11erx
D2X-XL Master
D2X-XL Master
Posts: 8112
Joined: Mon Sep 24, 2007 8:48 pm
Location: Wilferdingen, Germany
Contact:

Custom music and playlists

Postby karx11erx » Fri Jun 20, 2014 11:27 pm

There has always been the desire for better custom music support in D2X-XL, and steel's recent proposition in that regard has led to the final universal solution to once and for all solve all desires D2X-XL users could ever have in this regard. :joy:

D2X-XL now supports a three stage custom music system:

1. Mission specific custom music from mods ("playlist.txt" in the mod's root folder)
2. General custom music supplied by the user via a playlist listed in d2x.ini
3. Default Descent 1 and 2 custom music in the folders music/d1 and music/d2 in the static data folder (game folder on Windows) ("playlist.txt" in each music folder)

If "playlist.txt" is not present, D2X-XL will look in the appropriate music folders for music files with the names "level01.ogg", "level02.ogg", etc. for normal and "slevel01.ogg", "slevel02.ogg" etc for secret levels. D2X-XL also tries to load menu ("title.ogg"), briefing ("briefing.ogg") and credits ("credits.ogg") music from the mod and default custom music folders. The user supplied menu, briefng and title songs can be specified in d2x.ini.

D2X-XL supports OGG and FLAC. MP3 support has been removed from D2X-XL as it has never been working well due to bugs in the libraries SDL_mixer uses for its playback.

I will provide archives with the Descent 1 and 2 music in ogg format together with default playlists.

Enjoy. :D

Edit:

To mark a secret level song in a playlist, prefix its name with a minus sign ("-").
User avatar
karx11erx
D2X-XL Master
D2X-XL Master
Posts: 8112
Joined: Mon Sep 24, 2007 8:48 pm
Location: Wilferdingen, Germany
Contact:

When the game briefly stutters ...

Postby karx11erx » Mon Jun 23, 2014 11:28 pm

Recently I had the irritating effect that the game briefly paused whenever I saw a new powerup or weapon in a level. I debugged left and right, I rewrote my internal profiling code to see what might be happening, but I just couldn't figure what was going on.

It was pretty clear that it must have something to do with game content being loaded on the fly while playing. After a long while it dawned on me that I had added a command line switch named "-preload_textures" to D2X-XL a long time ago, allowing to control what game elements got loaded before the level started, and which got loaded on the fly. I had done that to decrease the game's memory foot print on computers with little RAM. In the course of that, I had also introduced another command line switch "-lowmem", making D2X-XL throw textures out of memory once they had been uploaded to the graphics driver. Again, I had done that to decrease D2X-XL's memory consumption.

Both switches can have annoying temporary effects on gameplay the cause of which may not be apparent to most of the D2X-XL users experiencing them. Heck, even I had to tinker around with D2X-XL for hours to finally figure what was going on! :lol:

Because of that, and since I reckon that every D2X-XL user has enough RAM these days in her/his computer(s), I have decided to remove these two switches. The recent bug fixes and improvements in D2X-XL's memory management have made that decision even easier.

This was one of the dumbest non-bugs I have ever been hunting. :crazy:
User avatar
karx11erx
D2X-XL Master
D2X-XL Master
Posts: 8112
Joined: Mon Sep 24, 2007 8:48 pm
Location: Wilferdingen, Germany
Contact:

Multi-threaded object rendering?

Postby karx11erx » Tue Jun 24, 2014 4:18 pm

A game can reap significant performance benefits from multi threading if it implements it right. Well ... D2X-XL didn't - at least not everywhere I tried to use it. The object renderer had been a very attractive target for making it multi threaded, because the required lighting calculations are quite complex and can be parallelized very well. So I had indeed tried to parallelize it. However, my implementation was severely flawed and caused the program to lock up. I had in fact written some very ugly code there, with a main loop constantly polling some global variables that ought to be set by the thread functions and vice versa, and that could just impossibly work, calling sleep (0) or even sleep (1) in between, which just kept the CPU busy for nothing really. At that time, I had tried to use SDL's semaphore implementation, but for some reason I hadn't figured how to apply them (simple as it is), so I had tried to accomplish my goal without them.

most of you probably won't ask why I didn't use OpenMP here, but I'll tell you anyway. :P OpenMP doesn't allow you to create some threads at some point of program execution and then keep them alive until program end. However, since object rendering is a task that gets repeated often, having to create the required threads every frame is an overhead I wanted to avoid. With SDL, I can create such threads and have them sleep when their work is done until they need to be woken up again to do some more work.

So I had to disable the entire multi threaded object renderer. I actually had never really forgotten about that. This had been a defeat for me, and defeats keep subtly preying on my mind.

Yesterday I came across that area of code again. I looked at it for a while, and it became abundantly clear to me why it didn't work. I needed semaphores to get it going. The documentation of SDL's semaphore implementation is very short, but it proved to be sufficient. I played around with different scenarios a bit, broke the renderer while doing that by moving it into the thread functions (which cannot work because only the program's main thread has a valid OpenGL context), fixed it again, and finally got it to work.

So what the object renderer now does is:

    1. Wake the object lighting threads up
    2. Sleep and wait for a lighting thread to tell it that is has something ready to be rendered
    3. Check whether all lighting threads have processed all objects assigned to them (by checking the active lighting thread count), and if so, quit
    4. Render the stuff
    5. When done rendering, wake the lighting thread up that had woken it up
    6. Go to 2.


What each lighting thread does is:

    1. Sleep and wait for being woken up
    2. Light a bunch of objects grouped together (by being in the same segment)

    3. Wake the object renderer thread up
    4. Sleep and wait for the object renderer to tell it to proceed

    5. If not done with all objects assigned to it, go to 2.
    6. Decrease the active lighting thread count. If the count goes down to zero, tell the render thread to wake up
    7. Go to 1

Since the renderer must not be called simultaneously by several lighting threads, the steps 3. and 4. need to be locked as soon as one lighting thread enters them, and need to be unlocked when the renderer is done.
Step 6 must be treated similarly. To avoid two lighting threads simultaneously accessing the active lighting thread count, that code needs to be locked as soon as one lighting thread enters it. Mutually exclusive pseudo code is marked orange, the other blue.

The result is a significant speedup when rendering many objects. In my test scenario I had the frame rate about doubled when using multi threading for rendering the objects. :clap:
User avatar
karx11erx
D2X-XL Master
D2X-XL Master
Posts: 8112
Joined: Mon Sep 24, 2007 8:48 pm
Location: Wilferdingen, Germany
Contact:

More stutter

Postby karx11erx » Wed Jun 25, 2014 12:22 am

Experienced stuttering again after entering a mine and found out that polygon model (ships, robots, powerups, ...) textures did not get pre-loaded ...

This has been fixed in v1.17.50.
User avatar
karx11erx
D2X-XL Master
D2X-XL Master
Posts: 8112
Joined: Mon Sep 24, 2007 8:48 pm
Location: Wilferdingen, Germany
Contact:

Post effects ...

Postby karx11erx » Wed Jun 25, 2014 11:33 pm

Post effects are effects that are rendered in screen space (i.e. the image that will appear on the screen will be changed). The Fusion cannon's shockwave ripple effect is such a post effect. These effects had not been working in side-by-side rendering modes (where the image is rendered two times on the screen, e.g. for 3D TVs). You didn't know that, did you? :mrgreen:

I knew it. All the time. But I never told you, hah! :P

People playing D2X-XL with the Oculus Rift might have noticed that (if they had cared enough about that effect). This evening, I finally had some time and motivation to look into that. I quickly found that rendering the screen image directly to OpenGL's hardware back buffer showed the post effects in side-by-side rendering modes. However, it didn't work when I was rendering the post effect shader result to another texture (FBO). I was trying to find out why for hours and hours. In a last, desperate attempt I explicitly activated a TMU before binding the shader program. Suddenly everything started to work.

Can anybody tell me why I did not have to do that for rendering to the hardware back buffer? :rant:
User avatar
karx11erx
D2X-XL Master
D2X-XL Master
Posts: 8112
Joined: Mon Sep 24, 2007 8:48 pm
Location: Wilferdingen, Germany
Contact:

It wasn't the driver ...

Postby karx11erx » Thu Jun 26, 2014 1:50 pm

surprise, surprise ... :P

Today, I found the real reason why post effect rendering could fail. Well, actually, I didn't find the reason, but I found out how to prevent it. That's good enough for now.
User avatar
karx11erx
D2X-XL Master
D2X-XL Master
Posts: 8112
Joined: Mon Sep 24, 2007 8:48 pm
Location: Wilferdingen, Germany
Contact:

Multiplayer ...

Postby karx11erx » Thu Jun 26, 2014 10:43 pm

Checked the multiplayer code this evening.

The code handling joining of players is broken. :horror:

I have no idea why. :think: The last time I remember playing multiplayer with D2X-XL, everything had been working fine.

Update:

I have been looking into the multiplayer code of d2x-xl now, and found a bunch of bugs. I have fixed what I have found. However, one issue that I need to test has been left: Apparently you cannot seamlessly transit into the next level in coop games if you are not the game host. I will look into that tomorrow.
User avatar
karx11erx
D2X-XL Master
D2X-XL Master
Posts: 8112
Joined: Mon Sep 24, 2007 8:48 pm
Location: Wilferdingen, Germany
Contact:

Coop problems

Postby karx11erx » Sat Jun 28, 2014 10:29 am

There have been reports of players being dropped from the game when transitioning from one level to the next in coop games.

I have been looking into this, and I have no clue what I had done in the past to make this happening. The most likely reason are some changes I made to make sure the game always had a game host. That was the short version. :P

When moving from a level to the next in multiplayer games, players who have left the level are placed in a score screen. The program will display that screen for a while, waiting for the other players to exit the level, before automatically proceeding to the next level. You can press a key to skip some or all of the waiting time.

While in that screen, the program checks whether any of the other players timed out, i.e. if no data has arrived from them for a certain amount of time. There are two scenarios where this can happen although those players have not actually left the game. In one scenario, a player has quit the score screen by pressing a key and his program is loading the next level. If this takes more than the timeout interval (just think of D2X-XL starting to compute lightmaps), the other players' programs will mark him as disconnected. If all other players are disconnected, the program will make itself the game host. (Actually it does so for the case that the game host actually leaves the game for some reason, to make sure there always is a game host.) Subsequently, it will not be able to reconnect to the real game host.

Another timeout scenarios may be players dying in the mine before reaching the exit. This is something I still need to test.

This was a bloody weird and hard to understand behavior.

I have started to make sure that all players who are still in the game but who are in some transitional mode not currently allowing them to routinely send "I am still here" messages to the other players will do so. Particularly the case where the program is very busy hasn't been solved yet. I know how to tackle this though, and it involves an extra network management thread. ;)

Return to “General Discussions”

Who is online

Users browsing this forum: No registered users and 1 guest