ryujinx
patreon
Progress Report April 2022
🕑 Added 2022-05-11 16:11:09 +0000 UTCApril… a month deriving from the Latin word ‘aperire’ or ‘to open’. How does this relate to Switch emulation? You can probably think of a few metaphors, but honestly I just thought it sounded kinda interesting. This month we’re covering some major changes and also a pretty meaty section on the recent progress to the Vulkan backend, which was absent last month.
Before all that though, check out our patreon goals and progress toward them:
Patreon Goals:
Amiibo Emulation
Merged into the main build in March 2021.
While compatibility is close to being perfect, there are still some improvements to come for Amiibo which can be tracked on the associated GitHub issue here: https://github.com/Ryujinx/Ryujinx/issues/2122
Custom User Profiles
Merged into the main build in April 2021.
Vulkan GPU Backend - still in progress.
A public test build is delivered and is available here! See the end of this month's report for some more details.
ARB Shaders - Goal reached in April 2021.
Work is ongoing alongside Vulkan, please wait a little while longer until we are able to deliver this update into a state we are happy with.
ARB shaders will further reduce stuttering on the first run by improving the shader compilation speed on NVIDIA GPUs using the OpenGL API.
$2000/month - Texture Packs / Replacement Capabilities - almost there!
This will facilitate the replacement of in-game graphics textures which enables custom texture enhancements, alternate controller button graphics, and more.
ETA once the goal is sustained: ~3-4 weeks
$2500/month - One full-time developer - Not yet met.
This amount of monthly donations will allow the project's founder, gdkchan, to work full-time on developing Ryujinx. All our contributors currently only work on the project in their spare time!
$5000/month - Additional full-time developer - Not yet met.
This amount of monthly donations will allow an additional Ryujinx team developer to work full-time on the project.
Alright. Let’s go!
GPU:
We begin this month’s report on a sad, sorry tale. Rune Factory 5 received its western launch in April and from everyone at Ryujinx we’d like to extend our thoughts and prayers to anyone who is yet to recover from playing… A moment of silence.
Become one with the void.
Awful art direction and thinly-veiled satire aside, the game did have a few graphical bugs at launch which were sourced to an error in the GLSL shader generation stages. Multisample and buffer textures do not take an LOD parameter, and so removing this from the GLSL generator for these particular stages resolves some of the missing effects that users reported.
Pinball FX3 was a title that worked flawlessly if you chose to play on the base version of 1.0.0, but suddenly just presented a black screen if any sort of update was applied. The cause of this problem was narrowed down to some troublesome multisampled 2D textures. If the texture to be copied did not exactly match the texture in the cache (for instance if the format or size changed), then this match would simply fail and cause a beautiful void of darkness. Allowing the copy texture view to not exactly match the cache here fixed the bug, and all four of the Pinball FX3 fans out there can enjoy the updated version!
Primitive restart, a vital feature in vertex rendering (lines and stuff), was somewhat broken in Project Diva Mega Mix but rather strangely was completely fine when tested against Vulkan. It turns out that in OpenGL the feature was being applied to both indexed and non-indexed draws, whereas in Vulkan the specification explicitly ensures only indexed draws make use of this. Luckily the GPU itself keeps a register that controls this behavior, and by reading this register as false for non-indexed draws, the issues can be resolved in a similar fashion to Vulkan.
Before
After
A less visual improvement was made this month with gdkchan laying some groundwork for some interesting titles to boot in the future. Both Super Mario 64, in the 3D All-Stars collection, and Perky Little Things would make use of enormous vertex buffers when booting and subsequently crash with memory-exhaustion errors. This change went through a few different approaches to the problem as far back as December, but now the vertex buffer sizes are reduced by calculating them from the buffer type. This brings them well within a comfortable range that Ryujinx doesn’t complain anymore. Further changes are required for these titles to boot but they aren’t far off now!
From the future: Between writing that sentence and when you (yes you!) are reading this SM64 does now boot! The changes will be covered next month but for now here’s a candid shot!
‘The cake was a lie’ - Princess Toadstool
Kirby and the Forgotten Land was fully playable on Day…. -200 and something (we tested on a few old LDN builds!) but this was not without a general spattering of minor graphical issues. Text and some other textures were noticeably wrong. Some were overlapping incorrectly, some were being cut off, menus could flicker etc. And for all that, the fix in total was just a single line of code. Checking if the graphical scissor was enabled before clearing was the key here, as before, a clear would take place regardless of if the scissor was enabled or disabled, thus causing some undue side-effects. These namely manifested themselves into all kinds of text and texture clipping, incorrect transparency and general wrongness!
Before
After
If you have a younger family relative, are a massive fan of a mega-franchise from George Lucas, or are head-over-heels in love with small pieces of colorful plastic, then this month also had a game for you! Lego Star Wars: The Skywalker Saga brought another OpenGL exclusive bug into the spotlight in the form of fragment clamping. Usually all fragment values are clamped within the range of [0,1] and normally this is completely fine but as we all know it’s never that simple, is it? LSW:TSS makes use of a specific data type called an ‘SNorm’ which actually has a full range of [-1,1] and so when these were clamped to [0,1] problems would arise.
Luckily, having isolated such a fundamental problem, the fix was also written and approved on Day 1 with these ‘SNorm’ fragments being exempt from the normal clamping restrictions. Character and environmental reflections are now rendered correctly and no longer make everyone look like Force ghosts.
Since riperiperi’s implementation of ‘Texture Sync’ a couple of months ago, Xenoblade fans have finally had to touch grass and submit some meaningful bug reports for the first time. Some nasty and, more annoyingly, random flickering and lighting bugs could be traced back to this change and resulted in the largest black market trading of old Ryujinx builds I personally have ever witnessed. While this chapter of the games lifecycle was rather amusing, riperiperi finally descended into the depths of dice-roll hell to try and crack down on whatever the new problem was. It turns out there were a couple of flaws in the original implementation mainly revolving around some sync methods exiting early without resetting the action flags. These may have caused the action to never be registered, and thus break any texture that caused this for the remaining time it is on-screen, or in the runtime until the game was reset.
MY EYES!
Forced to redact the party in Paint on spoiler grounds…
By resolving these edge cases and giving some areas a general re-order the main regressions caused by the original change should overall be a thing of the past once again!
It’s fixed now. Source: trust me bro.
AFTER
Some smaller GPU changes also took place this month with the VMAD shader instruction being implemented by gdkchan, mainly used in homebrew applications that utilize Nouveau OpenGL as the guest API, merryhime took a crack at optimizing the Lop3Expressions and a general de-cluttering of the graphics abstraction layer (GAL) was undertaken.
Alright with that out of the way let’s talk about some game changers.
Shader Cache 2.0:
Shaders are an ever-present and immutable fact of modern system emulation. They exist, they make things look pretty, but as all of you are likely aware, they don't play nicely on a system they weren't designed for. While Ryujinx can translate these shaders into something your PC can understand, this process takes time, usually longer than the render-time of the frame, which causes stuttering. Project Salieri, our first implementation of a ‘Shader Cache’, managed to mitigate most of the problems a user would face when encountering the same shaders for a second time. That was in 2020 though. Games in 2022 are more commonly using shader types and specialization that the original cache just wasn’t designed to deal with and in these scenarios it is left solely up to your GPU driver to remember what it's already seen. This isn't ideal as driver caches are prone to invalidation and effectively start from scratch at every driver update. Shader cache 2.0.... take the stage!
This update, delivered by Ryujinx's creator gdkchan, aims to solve multiple issues with the original implementation, including but not limited to: smaller cache sizes, shaders using bindless textures now being cacheable, shader specialization and some other quality-of-life changes like being able to close the program while shaders are being loaded and faster shutdown times if shaders have been cached in a session.
Shader specialization fixing Yokai Watch 1 flickers:
Before
After
The following (non-comprehensive) list of notable games can now cache a bulk of their shaders:
- Shin Megami Tensei V
- Mario Party Superstars
- The Witcher 3: Wild Hunt
- Pokémon Brilliant Diamond and Shining Pearl
- Lego Star Wars: The Skywalker Saga
- Many more both past and future…
BDSP specifically is a unique case where the game seems to use over 2000 shaders just to boot. These could previously not be cached, and due to the rather sluggish nature of the OpenGL shader compiler, users have noticed since release that these games took an indecent amount of time to boot. With the new cache, from the 2nd boot onwards this pain will be heavily reduced; we noted boot times reducing from over 80 seconds to under 15 with a full shader and PPTC cache!
Wait for it… Winner: New Cache
There are other advantages to the new cache format too. One of the biggest being it lends itself well to flexibility between graphical API’s. This means that when Vulkan is integrated the cache formats can be almost identical in nature and this will be touched on more at the end of this report!
CPU:
While April was undoubtedly the month of the GPU, our CPU backend still received a couple of important improvements with a major fix in regard to Amiibo usage.
Merryhime implemented the T32 load/store single instruction set which is another step on the road to ‘Ni no Kuni: Wrath of the White Witch’, and perhaps some unknown others, booting. More changes are needed to get in-game but with each new error we’re ironically one step closer!
Amiibo emulation is in a reasonably solid state, but in a few titles users were experiencing massive slowdowns when the menu was opened, and these would persist until the game was hard reset.
This was a regression from a prior PR that optimized the tail merge passes and went under the radar for nearly a year. gdkchan stepped up to add additional checks in the tail merge methods which resolves these slowdowns. Some of the impacted titles include: Animal Crossing New Horizons, Kirby and the Forgotten Land and potentially other titles when used with Amiibos.
Pictured: Kirby demanding new plastic toys!
AUDIO:
Surprisingly popular sports title: ‘MLB The Show 22’ launched in April and unfortunately it isn’t currently in-game. However, similar to Ni no Kuni above, the steps toward this have already begun with the first being the implementation of multistream related Opus decoding functions in the audio service. Opus, according to their own webpage, is a “royalty-free, highly versatile audio codec… unmatched for interactive speech and music transmission over the internet” so there may be a considerable number of past and future titles that can make use of this addition.
Our audio renderer, Amadeus, also received two updates this month. The first was the laying of the boilerplate for the newest revision of the renderer, REV11, by Thog. This revision came with firmware 14.0.0 and changed the channel disposition for legacy audio effects such as: Delay, Reverb and Reverb 3D. Followup work is planned to fully re-implement these effects, but for now this change redirects them to the legacy system as a temporary solution.
The second change is directly linked to what was just mentioned with Thog starting the work to re-implement these functions, beginning with improvements and fixes to delay effect processing. No changes are expected in games with these audio adjustments, but the hope is audio-related problems in future titles can be avoided should they use modern revisions of the renderer.
SERVICES/MISC:
First-time contributor german77 resolved an issue in ‘Flip Wars’ where the game would insist that the controller was constantly disconnecting, while users were left baffled as other games didn’t exhibit this behavior, and as far as they could tell their controller was set up completely normally.
It turned out that certain games expect an AcquireNpadStyleSetUpdateEventHandle signal to be called during gameplay regardless of the status of the controller connection status. By returning this event, the random controller disconnects in this title were resolved and the game is now fully playable.
Input improvements continued with a second first-time contributor (but full-time complainer) Haronee fixing a long-standing bug in the native motion controls implementation. Users had noted that the axis that they held was oftentimes inverted, and if they held left they’d actually receive right or vice versa.
It turns out the problem really was as simple as it sounds. The Z axis was incorrectly attributed as positive when it should have been negative. This sign flip fixes issues in games where motion controls using the native setting (not cemuhook) would be backwards.
LibHac, the FileSystem service we utilize, also received a version bump this month to 0.16.1. This update fixes a regression where NSO titles such as NES and SNES online would crash on startup, adds support for reading XCI files that contain an initial data/key area and finally adds key sources for firmware 14.0.0 onwards. It is worth noting that this is required for FW 14.0.0 upwards to function, and be warned that if you share firmware between master and the older LDN build, LDN will start to get very angry and write lots of red text in the console!
To wrap up this main section we’ll enter the quickfire changes round:
- The prompt that asks if you wish to create save data if none exists has been removed to streamline the experience.
- An edgecase in ReactiveObject has been addressed to handle when a value becomes NULL.
- Ryujinx is now DPI aware on linux systems using X11 to display the program. This can be overridden by Xft.dpi settings.
VULKAN PROGRESS:
Ah, the section everyone seems so very interested in. This is my version of “watch to the end of the video!” in text format. Luckily this isn’t clickbait and we have a lot of progress to share with you on what is quickly turning out to be an AMD ‘Whack-a-mole’ experience. One bug gets squashed and three more games start to flicker! Before that though let’s talk about what’s going well.
NVIDIA Supremacy
A significant milestone was reached this month, with almost every title that the testing team possessed being functionally equivalent on Vulkan using SPIR-V and OpenGL using GLSL. So let’s dive into some titles that were once borked and are now not-borked.
Skyward Sword HD:
Ugly brown duckling no more!
Pokémon Brilliant Diamond and Shining Pearl:
These cash-grabs also finally boot! Rejoice AMD and Intel stans.
Luigi’s Mansion 3:
Virtually every major title we threw at it is now on equal footing to the OpenGL backend, but this comes with the caveat of saying ‘on NVIDIA GPUs’. More work is needed to fix some rather ridiculous bugs team red seem to be conjuring from the depths of hell each passing day.
The heat has made Mario rather prickly lately.
What does AMD have against Italian plumbers?
As usual if anyone works for/knows someone who works for AMD: can you take every opportunity to remind them that an Intel iGPU from 2017 is somehow causing less headaches than some flagship Radeon dGPU’s? Looking at you Polaris!
Shader Cache 2.0 continued:
Back to some good news, earlier in this report the rewrite of the shader caching system was discussed at great length, but details on how this would impact Vulkan were purposefully omitted as the changes here are extensive and part of the reason many more games are functional.
Shortly after the merge of the cache rewrite, gdkchan added the new system to Vulkan, which means that both multithreaded SPIR-V compilation and shader caching is fully functional across all vendors! Exciting stuff even for our testers.
Indirectly, the new cache also fixed some Vulkan-exclusive graphical issues, particularly when multisampling was used to render the image. The two most common issues were from Super Smash Bros: Ultimates title menu and Pokémon Legends Arceus’ town and battle style cards.
Before
After
Before
After
Before
After
That's a wrap! Some of the eagle eyed amongst you may have noticed how seamlessly we skirted around the mention of the newly released Nintendo Switch Sports, and to that I raise a glass. Unfortunately, it did break our rather considerable streak of day 1 playable first-party titles that dates back to sometime in 2020, but progress is being made! The game boots on our master builds but will promptly crash shortly after reaching the menus. gdkchan, juggling around five other projects, found the time at launch to get some fixes in place, which we hope can be finalized soon! Check out some horribly compressed footage below (spot the JoyCon drift challenge):
As usual, if you’re interested in emulation and know some C#, we’re always delighted to see new code contributors who can tackle anything from typos to rewriting entire systems! If that sounds above your paygrade then simply giving us feedback, opening issues on GitHub or just reporting compatibility helps us out enormously too!
See you soon!
