The Force Engine (Jedi Engine Port/Replacement)

[holaareola]

This is incredibly cool. Look forward to when the UI / briefings etc are done, I'll have my first playthrough in a couple of decades. And probably quit in rage once again at the dark annoying sewer level.

[MrFlibble]

The sewer is nuthin' compared to the Kell dragon fisticuffs IMO.

[lucius]

4 hours ago, holaareola said:

This is incredibly cool. Look forward to when the UI / briefings etc are done, I'll have my first playthrough in a couple of decades. And probably quit in rage once again at the dark annoying sewer level.

The sewer level is pretty straightforward when you get used to it. To do it quickly, when you get to the 4-state switch, go through the first door (hitting the switch to raise the sewer level during the process), then the third door, and finally the second door. That will save you a trip through the tunnels. You can ignore the fourth door unless you want the goodies there (but I don't bother when I'm playing through it).

 

I should add - you can hit F5 to turn on your headlamp to help illuminate dark areas. It consumes battery power, but there are battery pickups and the consumption is reasonable (unlike some games where flashlights last for two seconds).

Edited November 18, 2021 by lucius

[holaareola]

38 minutes ago, lucius said:

The sewer level is pretty straightforward when you get used to it. To do it quickly, when you get to the 4-state switch, go through the first door (hitting the switch to raise the sewer level during the process), then the third door, and finally the second door. That will save you a trip through the tunnels. You can ignore the fourth door unless you want the goodies there (but I don't bother when I'm playing through it).

I'll make a note. I remember really enjoying DF other than that, but I hated, detested, wished I could have a violent personal intervention with the designer for that map.

 

Off-topic, if I could, I'd expunge sewer levels from FPS history, it has never struck me as an interesting environment. Traversing a bunch of small-ish, pitch dark passages, literally swimming through virtual shit as a bonus of the setting. I honestly think the preponderance was due to light asset requirements and an excuse for really repetitive design. The utterly excrementitious hollywood sewer section in Vampire Bloodlines remains my all time #1 most hated. Mirror's Edge has the #1 most loved sewer, taking the top spot only due to there being NO OTHER CONTENDERS.

 

Anyway, I feel lighter, cheers for the advice and your effort on the port. I remember there being quite a few level packs/mods out for DF too, will they be compatible?

 

 

3 hours ago, MrFlibble said:

The sewer is nuthin' compared to the Kell dragon fisticuffs IMO.

I don't remember this. Urk. Doesn't sound good. Melee was yet to become a strong point in FPS games for a long time after this was made!

Edited November 18, 2021 by holaareola

[lucius]

9 minutes ago, holaareola said:

I remember there being quite a few level packs/mods out for DF too, will they be compatible?

Most mods work, though support is "beta" quality - meaning some still have TFE-related bugs and a few can crash at a certain point. But the plan is that they all work before version 1.0 is released. I will be making a post, most likely later tonight, about the mod support and mod loading.

Edited November 18, 2021 by lucius

[Jello]

2 hours ago, lucius said:

The sewer level is pretty straightforward when you get used to it. To do it quickly, when you get to the 4-state switch, go through the first door (hitting the switch to raise the sewer level during the process), then the third door, and finally the second door. That will save you a trip through the tunnels. You can ignore the fourth door unless you want the goodies there (but I don't bother when I'm playing through it).

 

I should add - you can hit F5 to turn on your headlamp to help illuminate dark areas. It consumes battery power, but there are battery pickups and the consumption is reasonable (unlike some games where flashlights last for two seconds).

This is some information that could've been brought to my attention YESTERDAY!

 

Quite literally, I decided to try out the work that you've done on Dark Forces, and it's absolutely great. Getting to run this smooth in Windows, and with resolution scaling, I mean it's absolutely amazing the work that you've put into it. I did play a little of the GOG version, but I haven't really played, and completed, the game since 1996. And that was on DOS, where it played just fine. So to be able to play it again on a modern system without having to dink around with Dosbox is amazing, so kudos to you. 

 

That aside, I decided to try it out last night, and I was stuck on that damn sewer level for at least an hour. I missed one switch towards the beginning, the one that raises the sewer level. It is so easy to miss switches in this game. Great game. But it seems like the level designers started out their dev map with "Piss the player off".

 

Still, very impressive what you've accomplished. I look forward to seeing the finished product, because it already plays really nicely.

[lucius]

After a long time in development, the Core Game Loop Release, version 0.7 is finally available. All of the weapons, items, and AI have been integrated. Dark Forces can be completed from beginning to end using The Force Engine. TFE continues to support the original, 320x200 fixed-point renderer but now properly supports high-resolution rendering, and widescreen (including Ultrawide). TFE can load mods directly from ZIP files, you can drag and drop them onto the executable. It also has a built-in Mod Loader, accessible from the main menu.

 

To get mods to show up, simply create a Mods folder in your TFE directory, the Dark Forces game data directory, or under /ProgramData/TheForceEngine. In that mods folder, you can put the zip files, one per mod, or extract them to one sub-directory per mod.

 

Below is a video that marks the release of version 0.7 - the Core Game Loop release. This video shows part of Detention Center, and then small parts of three user levels from “back in the day.” Note that mod support is currently in “Beta”, not all mods fully work, and some may even crash. However, these issues will be resolved in future versions. There are still bugs and issues, but the Dark Forces can be finished from beginning to end using TFE.

 

 

What’s Next

Now that the CGL release is done, it is time to move to the next milestone on the Roadmap - Cutscenes, Mission Briefings, and completing the in-game UI (PDA). That will be version 0.8. The rest of this year is dedicated to finishing versions 0.8, 0.9 (sound and iMuse), and finally version 1.0.

 

https://theforceengine.github.io/Roadmap.html

Edited November 19, 2021 by lucius

[lucius]

Version 0.8 is well underway and should be complete this week.

 

Cutscenes in the vanilla game all play, though there are some visual glitches to iron out.

 

 

The Mission Briefings and PDA are also complete - along with fixing a bug with difficulty levels and finally loading and handling mission "goals" correctly - so they show up properly in the PDA when completed.

 

 

So what is left for version 0.8:
* Fix the font in the Agent Menu (some characters show up wrong).

* Add the keyboard shortcuts to the Escape Menu. I will probably not do the fade, despite the small inaccuracy, because 1) it isn't complete and looks kind of bad, 2) I don't want it to take so long when hitting Escape and being able to use the menu. Same with the PDA - you hit the button and it comes up near instantly.

* Figure out what to do with the config menu - I'm either considering implementing it as-is or having it bring up the Alt+F1 menu instead - where all the valid configuration options are available. The latter option is what I am leaning towards since it will be less confusing than having a partial set of options, some of which are useless right now (the level of detail options).

* Fix visual glitches in cutscenes.

* Integrate sound and music support for cutscenes (they are currently silent).

 

Another nice feature is that TFE supports automatically loading custom briefing and cutscene LFD files for mods, so players can load their mod directly from the zip files and TFE can still support the custom mission briefings and cutscenes.

Edited December 2, 2021 by lucius

[lucius]

The Cutscenes and In-Game UI Release build is now available: https://theforceengine.github.io/downloads.html

 

 

This video marks the version 0.8 - Cutscenes and In-Game UI Release. The cutscenes are now played, even from mods, with sounds and music. The Mission Briefings are integrated, including the ability to pick the desired difficulty. In game, the PDA (F1) is complete, allowing you to see your inventory, weapons, the map, mission objectives that you have completed and secrets found, and the briefing. Finally, the "Configuration" option in the Escape menu now opens the TFE System UI with full resolution and keybinding menus.

 

This video shows the initial cutscenes, the mission briefings for a few levels, config options, cutscenes in a mod, and the mission briefing in another mod. Note: iMuse is only partially reverse-engineered, so music in the cutscenes is not 100% accurate - this will be fixed in the 0.9 release coming in the next few weeks.

 

Next is the sound and iMuse release, version 0.9, at which point Dark Forces support in TFE will be feature complete. However, there are still some bugs and mistakes, which will the focus for version 1.0.

[lucius]

 

Version 1.0 Release Delayed

This is obvious by now, but The Force Engine version 1.0 will not hit this year. There has been a lot of progress towards completing the iMuse reverse-engineering work for version 0.9, but that will spill into early January.

 

2022 Plans

The iMuse work is roughly 70% complete, which means that version 0.9 is expected to land in the second week of January. At that point, Dark Forces support in TFE will be feature complete and the reverse-engineering process for Dark Forces will be finished. The following few weeks will be dedicated to bug and inaccuracy fixing, with version 1.0 planned for late January or early February.

 

To read more about 2022 Plans and the 2021 Retrospective, read the post: https://theforceengine.github.io/2021/12/24/2021-retrospective.html

[liPillON]

hi Lucius, I hope all is well

I'm a bit worried, seeing as the project's website hasn't been updated since xmas..

how's the progress on this?

 

[JPL]

5 hours ago, Delfino Furioso said:

hi Lucius, I hope all is well

I'm a bit worried, seeing as the project's website hasn't been updated since xmas..

how's the progress on this?

 

 

If you want to keep tabs on the very latest work done, see this view of the different branches in github:

https://github.com/luciusDXL/TheForceEngine/branches

Last commit was a couple weeks ago. Also worth noting that some work might be going on in non-public branches or depots.

This is really normal with spare time open source projects, people get pulled away by real life stuff, energy for the project wanes and waxes, etc. I say let folks take a break when they need to. Lucius made incredible progress over the last year, I'm happy to wait.

[Dzierzan]

He's still working on the IMUSE. Keep in mind that he also has a private repository where he commits reverse engineered Dark Forces code.

 

Also, have this:

 

Edited February 17, 2022 by Dzierzan

[lucius]

Sorry for the lack of updates on the forums, but work continues on iMuse, as Dzierzan says. At the pace of work last year, it would only have taken a few weeks to complete but unfortunately, there has been less time available at the beginning of the year (Q1 issues). The reverse-engineering for the midi part of iMuse is basically done (it also handles digital sound effects to some degree, especially in cutscenes) but the reconstructed iMuse library that will be used by TFE is still a work in progress. You can see the post linked above by Dzierzan for more details.

 

Also, I do keep a private repository for reverse-engineered code, "Dark Forces DOS" - it is more of a "code document" than something I would ever release, with a lot of extra data, such as memory addresses and notes that don't appear in the final code. The refactored, cleaned-up version of the code is what makes it into TFE. The private RE "code document" has extra bits that are handled by the TFE framework - such as VGA code, various interrupt handlers, low-level DOS audio driver code, etc. It also contains things like unused structure members (without proper names because I don't know their original purpose), empty functions (code compiled out), wrapped functions that serve no purpose (i.e. whatever the original purpose was, it was later removed or compiled out, such as debug code), and that kind of thing that is cleaned up for TFE. But to give you an idea why I would never release it, one of the files is over 60k lines long.

Edited February 17, 2022 by lucius

[lucius]

Version 0.9 - iMuse and Sound - has been released.

Version 0.9 has been a long time coming, taking much longer than originally anticipated. Version 0.9 is focused on sound playback and music accuracy, ambient sound support, and a few bug fixes.

 

Features and changes included in the release:

• Reverse-engineering of Dark Forces complete (with the obvious exception of going over bits of code again for bugs, or if I ever want to port TFE to DOS - and no that is not happening anytime soon if ever).
• iMuse system with proper music cues, transitions, and effects. Features like fades and similar effects work the same for both midi and digital audio.
• This is where most of the time was spent. iMuse is big and convoluted.
• Game music with fight/stalk transitions.
• Game and Cutscene sound system that uses iMuse to play digital audio; which includes proper sound priorities and accurate sound falloff and panning.
• Level ambient sounds.
• Sound UI will volume control for Cutscene Music, Sound, and Game music and sound.
• The ability to enable 16-channel digital audio support in iMuse (it was basically already there in the code, I just had to make some tweaks so it could be changed at runtime). 8-channels is the default like DOS but you can enable 16-channel support in the Sounds config.
• The ability to disable fight music if desired.
• The Mousewheel is now bindable.
• The Mousewheel works on mission briefings and some PDA screens (mission briefing and map).
• Improved support for System UI scaling for 1440p and 4k.
• Many other fixes.

Full blog post: https://theforceengine.github.io/2022/05/17/iMuseAndSoundRelease.html

Downloads: https://theforceengine.github.io/downloads.html

 

Version 1.0 Plans

With version 0.9 finally released, the next major release will be version 1.0 - complete support for Dark Forces in TFE. Unlike the 0.9 release, the plan is to split up the release into several smaller releases. There will be 2 main parts:

• Bug Fixes. I plan on splitting bug fixes by system and doing one or more 0.9x releases for each system. Examples include AI bugs, weapon bugs, collision issues, INF issues, etc.
• The GPU renderer - the last major feature for version 1.0. The GPU renderer will support both shearing for looking up and down (which emulates the software renderer) and accurate perspective projection to allow the player to look up and down further without distortion like in modern 3D games. Initially, it will support 8-bit color emulation with optional colormap interpolation to remove banding. Later, after version 1.0 is released, true color rendering, dynamic lights, and other features will be added.

 

 

Edited May 18, 2022 by lucius

[Kinsie]

I don't seem to be getting any music with the new release.

[lucius]

26 minutes ago, Kinsie said:

I don't seem to be getting any music with the new release.

That is definitely odd. You can check settings and make sure the volume isn't set to 0. You can also go to Documents/TheForceEngine and delete settings.ini, so TFE will regenerate it on the next load and see if you get music under default settings.

 

A few questions: 

1. Do you get any music during cutscenes? Or no music at all?

2. Do you hear sound effects, or is it completely silent?

3. Try deleting the settings.ini and then check again and see if that fixes it.

4. You can check the log, Documents/TheForceEngine/the_force_engine_log.txt, and see if there are any midi-related errors printed in there.

5. TFE uses General Midi, so make sure that isn't disabled somehow (or the volume manually turned down in Windows). Though there doesn't seem to be any obvious way to access this anymore in Windows 10+. :(

Edited May 18, 2022 by lucius

[Kinsie]

13 minutes ago, lucius said:

That is definitely odd. You can check settings and make sure the volume isn't set to 0. You can also go to Documents/TheForceEngine and delete settings.ini, so TFE will regenerate it on the next load and see if you get music under default settings.

 

A few questions: 

1. Do you get any music during cutscenes? Or no music at all?

2. Do you hear sound effects, or is it completely silent?

3. Try deleting the settings.ini and then check again and see if that fixes it.

4. You can check the log, Documents/TheForceEngine/the_force_engine_log.txt, and see if there are any midi-related errors printed in there.

5. TFE uses General Midi, so make sure that isn't disabled somehow (or the volume manually turned down in Windows). Though there doesn't seem to be any obvious way to access this anymore in Windows 10+. :(

Poking around, it looks to actually be an issue on my side. Sorry for the hassle!

[lucius]

3 minutes ago, Kinsie said:

Poking around, it looks to actually be an issue on my side. Sorry for the hassle!

Good to hear, thanks.

[lucius]

0.9x Bug Fix Release 1

TheForceEngine-v0_90_000-13-g2b1bc8d: https://theforceengine.github.io/downloads.html

• Fixes a sound crash when exiting to the main menu while in a level and then starting up a mod.
• Fixes a Dark Forces bug where music doesn't play in a cutscene on a second viewing in the same session.
• Fixes a bug when an object has an incorrect class or type on load - it should be skipped.
• Clean up differences between TFE level loading and DOS.
• Fixes a bug where the ambient sound index was always 0 (incorrect ambient sounds could play).
• Correct screen flash fade lengths for shield and health damage, and pickup flashes. Before some effects were not long/intense enough, especially when taking large amounts of damage.
• Fixes crash when a destructible object has no animation 1 - this resulted in numerous crashes (though almost all in mods).
• Fixes the sound update when a projectile is passing by the camera.
• Fixes an issue that could cause textures to fail to load if one line is malformed.

 

Please see the post above about the iMuse and Sound 0.9 Release above if you haven't already. As I talked about there, I plan on posting semi-frequent 0.9x bug fix releases leading up to the version 1.0 release.

Edited May 19, 2022 by lucius

[lucius]

0.9x Bug Fix Release 2

TheForceEngine-v0_09_000-15-gc28638a: https://theforceengine.github.io/downloads.html

• Fixed OBJ_FLAG_HAS_COLLISION, which should be OBJ_FLAG_MOVABLE. This fixes errors such as corpses not moving with rotating sectors.
• Fixed object collision radii - this fixes a bug where the collision for many objects was too large.
• Fixed a bug where TFE crashes if there is no briefing.
• Fixed a bug with collision where a flag was incorrectly checked - it should only check the collision radius. This fixes a bug where some props did not have collision when they should have.
• Fixed the default message type for line triggers (which can affect some INF functionality in mods).
• Fixed several areas while loading INF that could cause crashes with malformed data.
• Fixed slave “value” - I thought it was fixed-point but it is actually an angle.
• Fixed triggers to accept event == INF_EVENT_ANY (aka -1), this can also have an effect on INF in mods.
• Fixed collision issues with moving and rotating walls.
• Fixed object default flags - this also could cause some objects to not move correctly with rotating or moving sectors.
• Fixes a bug when decompressing transparent textures (compressed transparent BM) - this fixes bugs such as holograms in Assassination at Nar Shaddaa.

[lucius]

0.9x Bug Fix Release 3

TheForceEngine-v0_09_000-27-g5f123b0: https://theforceengine.github.io/downloads.html

 

• Fixes a crash that occurs when a cutscene “film” fails to load.
• Fixes a crash when the top or bottom texture of a wall is null.
• Frees a digital sound if it fails to set up data, which avoids a bug where all of the channels can be consumed.
• Fixes a bug where a teleport can fail to be set up correctly if it has an invalid field - this caused a crash in Dark Tide 2 when going into water.
• For Boba Fett, make double sure the msg is MSG_RUN_TASK before freeing the task after death - which can cause a crash on DT 2.
• Added support for half-step rounding when a rotating elevator delta is too small (due to high framerate combined with low speed). - This fixes the train in Harkov when running at higher than 90 fps.
• Fixes a bug where a texture with a blank name wasn’t set to default - some mods, like the Dark Tide series, such blank texture names to mane default.bm for some reason. This fixes missing textures and HOM effects.
• Reduces the velocity jitter threshold by half so the player doesn’t get stuck on low friction surfaces when the framerate is high. - This fixes a bug where I got stuck underwater in DT 2 with vsync off.
• Made time tracking more robust - Ensure that time is purely monotonic; disable vsync interval rounding at 120+ fps since it becomes unreliable; recheck sync and refresh rate less often to avoid stutters.
• Fixed a bug where some trigger functions were firing even with the trigger master was off.
• Fixed a potential crash when iterating through objects in a sector.
• Reduced external velocity 0 clamping by half to better handle high framerates. - This should reduce sliding on some moving platforms at high framerates.
• Cleaned up landing animation code, which is now accurate.
• Cleaned up head/weapon wave code, which is more accurate.
• Fixed the headlamp calculation which was slightly inaccurate.
• The cutscene music repeat bug is fixed when canceling from the mission briefing (previously this bug was fixed when entering a mission, exiting out, and then playing the cutscenes again. This bug handles the case where you cancel from the mission briefing).
• Fixes a crash in DT 1 because sector vertices are NULL.

[lucius]

0.9x Bug Fix Release 4

TheForceEngine-v0_09_000-34-gdb7a5c0: https://theforceengine.github.io/downloads.html

 

• Fixes Kyle incorrectly sliding off of moving platforms in some mods, like Dark Tide 1.
• Fixes a bug with animated textures that were not setup correctly - this fixes some switches that did not show up at all in mods, and potentially other animated textures.
• Fixes a bug where transparent animated textures might show up as opaque, this affected some mods and switches.
• Fixes a bug where animated texture frames may have a few incorrect/corrupted colors.
• Fixes a bug where some switches could not be triggered in mods, and other switches were harder to trigger than they should be.
• Added a ‘warp’ command to the console so you can warp directly to positions reported by the LADATA cheat.

Edited May 25, 2022 by lucius

[lucius]

0.9x Bug Fix Release 5
TheForceEngine-v0_09_000-42-g32a941a: theforceengine.github.io/downloads.html

• Fixes additional sign rendering issues if the base wall texture height is smaller than the switch texture height.
• Fixed an issue where some switches couldn’t be triggered.
• Fixed a crash if warping to an invalid position.
• Fixed crash that could be caused by an elevator with a null sound, which caused a crash when blowing up the lava dam in the Lava Planet mod.
• Fixes bugs with 3DO parsing, which fixes missing 3DOs in mods.
• Fixes a display issue with “tall screen” resolutions (i.e. taller than 4:3).
• Fixes a bug where elevator should remove corpses and continue instead of hitting them and moving back up.
• Fixes a bug where an elevator thinks there isn’t enough room to fits objects incorrectly, which fixes the bug in DT 3 where the blue key rises, hits the ceiling and then lowers again.
• No longer sets 0 channels even when requested - fixes music stopping permanently in Imperial Library.
• Verifies proper texture frame sizes and uses frame 0 if a animated texture asset is bad - fixes crash in Imperial Library.
• Fixes a crash that can occur if a mid-texture isn’t setup correctly on a solid wall - this fixes the Dark Tide 3 “Droid Deception” level end crash and visuals.
• Fixes looping VUE animations (such as Nar Shaddaa and Executor).
• Fixes incorrect 3DO orientations (mostly noticeable in mods).

Edited May 27, 2022 by lucius

[lucius]

0.91 Bug Fix Release 6

TheForceEngine-v0_91_000: https://theforceengine.github.io/downloads.html

• Added optional extended adjoin limits - this fixes HOM issues in most mods due to engine limits if enabled (default).
• Added optional crosshair support (defaults to off).
• Fixed 3DO lighting to match DOS.
• Fixed a 3DO gouraud shading clipping bug when using the floating-point renderer (which caused streaking artifacts).
• Fixed differences between DOS and TFE 3DO/sector clipping.
• Fixed the pitch offset calculation (it was slightly off before) - this makes shots fired up and down more accurate.
• Improved Escape menu button hover offsets.
• Fixed a bug where it was possible to not have a mission selected on the Agent Menu, causing a crash if beginning the mission.
• Fixed a bug where completion of the final level wasn’t shown on the Agent Menu.
• Fixed a bug where level complete status wasn’t saved when hitting Quit to DOS instead of Next Level.
• Fixed a bug where land mines were triggering through doors and similar situations.
• Fixed a bug where death effects kept occurring even after reaching 0 health - this fixes the repeating death sounds on damage floors.
• Enemies now die properly if they fall too far.

This build fixes 3DO rendering issues, various mission complete issues, dying effects (such as repeating sounds), and various other bugs. It also adds the ability to use crosshairs - with custom color, scale, and 18 patterns to choose from. And finally, with the optional extended adjoin limits, HOM issues in mods are greatly reduced and in many cases eliminated.

 

Extended Limits - Before/After (Dark Tide 3)

 

Crosshair Support

 

Next Steps

Many issues have been resolved to get to version 0.91. While a few INF and AI issues, as well as a possible rendering issue and various minor issues remain to be fixed for version 1.0; I am planning on taking a break from the bug fix releases to finish up the GPU renderer. The plan is to finish the renderer and get it released, then go back and finish up the last issues for version 1.0.

[lucius]

GPU Renderer Update:

 

The GPU renderer has been progressing nicely. Recently lighting and texture offsets have been implemented, leaving "signs", animated textures, and sky handling to complete the core sector renderer. You can look nearly straight up and down but there will be optional pitch limits to match closer to the original and work better with sprites and skies. (Note: the last image was captured before texture offsets were implemented).

 

 

 

Edited June 7, 2022 by lucius

[liPillON]

beautiful!

 

[lucius]

Another quick update - work is progressing with proper sky rendering support for the GPU renderer. This is obviously a challenge since the original skies were not meant for perspective when looking up and down. I am currently working on properly handling adjoins with skies (so you will notice some walls that should be sky).

 

 

 

 

 

[lucius]

I uploaded a video showing the GPU sector renderer in action

 

 

The core features for the sector renderer are done, though I still have some work to do on sky/pit adjoins and some minor issues (mainly the black flicker you get every once in a while because the portal near plane check is too aggressive). At this point, I am going to go on to object rendering and then sweep back to those issues afterwards.

[lucius]

The previous video showed a first look at the TFE GPU Renderer, which showed rendering sector geometry but no objects or HUD. This time, I run through another level to show objects, the HUD, weapons, and the automap. There are still a few issues here and there, but the GPU renderer is getting much closer to the finish line.

 

Of course, the software renderer will be maintained. But the GPU Renderer offers much higher performance and correct perspective rendering when looking up and down. Despite what the UI says, the renderer uses 8-bit palette emulation.

 

Note - the remotes are a lot harder to hit without auto-aim. :)-