Android: add support for content:// directory traversal

[crudelios]

• I confirm that I am the author of this code and release it to the SDL project under the Zlib license. This contribution does not contain code from other sources, including code generated by a Large Language Model ("AI").

This PR adds support for traversal of content:// directories on Android, as well as getting SDL_PathInfo on content:// files.

Description

In order to allow proper traversal of directories and keeping platform compatibility, I had to come up with some changes for the way SDL handles URI's obtained from the "folder select dialog" on Android,

The problem

A normal use case for file access on games is for users to set a base path (such as the install path), then get subfolders and files from that path.

So, if the basepath is something like:

C:\Programs\MyGame

The library user tends to append the subfile to that. Example: appending /mysubfolder/myfile.txt to the base path of C:\Programs\MyGame, which results in C:\Programs\MyGame/mysubfolder/myfile.txt.

This works fine on most platforms, but not on Android with content:// URIs.

Android content:// URIs are part of the Storage Access Framework (or SAF), and may reference a tree (a root path), a document within a tree or an individual document. Either way, due to the way the directory is structured, simply appending /mysubfolder/myfile.txt to a content:// URI will not work. Worse yet, it will appear to work but will point to a completely unrelated document, such as the root directory of the tree (i.e. the folder selected from the dialog).

An example of a tree URI (user selected <external storage>/folder/subfolder from the "folder select" dialog).

content://com.android.externalstorage.documents/tree/primary:%3Afolder%2Fsubfolder

Appending /mysubfolder/myfile.txt to this URI results in:

content://com.android.externalstorage.documents/tree/primary:%3Afolder%2Fsubfolder/mysubfolder/myfile.txt

Such an URI will actually still point to the <external storage>/folder/subfolder directory and not to the selected file!

To get the file, the user would either need to:

• Manually query Android's SAF for the URI of the file he wants (a lot of work for a regular user, but that's the implementation of this PR); or
• Manually create the document URI and hope it worked.

For reference, a proper document URI for the musubfolder/myfile.txt using the above tree URI would be:

content://com.android.externalstorage.documents/tree/primary:%3Afolder%2Fsubfolder/document/primary%3Afolder%2Fsubfolder%2Fmysubfolder%2Fmyfile.txt

This is very error-prone and Android-specific, demanding #ifdefs for Android in the user code. Worse yet, the URI syntax may change in the future, rendering the user's code unusable.

My solution

I wanted to create a solution that was:

• Performant (as performant as handling Android SAF can be)
• Portable/easy to use
• Still compliant with original content:// URIs.

What I came up with was to change the URI provided by SDL_ShowOpenFolderDialog on Android.

WIth this PR, SDL appends a fragment separator (an #) to the end of the provided URI. This will be ignored by Android (so the user can directly use the provided URI) but will be picked up by SDL's functions. Similarly, content:// URI's without # are still compatible with SDL - as long as they point to a valid file.

Whathever comes after the # on the URI will be treated by SDL as the subpath inside the directory provided by the tree URI.

So in our example above, if the user wanted to access mysubfolder/myfile.txt, he really would only need to append that to the URI, as such:

content://com.android.externalstorage.documents/tree/primary:%3Afolder%2Fsubfolder#mysubfolder/myfile.txt

SDL will properly navigate the provided path tree and find the file requested by the user, obtaining the correct URIs that are provided by the SAF API.

Additionally, since SAF traversal is slow, the provided results are cached in memory so subsequent accesses are much faster.

In order to allow proper original content:// directory enumerations, I've also added functions that convert tree URI's to document URI's, ending with a proper path separator (a %2F), in order to comply with the dirname specification of the SDL_EnumerateDirectoryCallback - in other words, even with original content:// URI's, chances are that if you append fname to dirname when enumerating a content:// directory and request info about the file, it will still work.

While I've decided to cache the SDL specific URI's, I've decided against that on original content:// URI's. The reason being that if the user is providing a clean content:// URI, he likely knows what he's doing and doesn't need SDL in the way.
Edit: I've decided to cache real URI's as well. I feel the performance difference is too high to just ignore. Due to the different way real URI's are cached, cached fetches for URI's are about 3x slower than for SDL provided URI's - however, they're still orders of magnitude faster than direct SAF lookups.

Conclusion

I'm sorry for the very long PR description, but I felt the need to explain the underlying issue and my solution.

I've tested most of the code and couldn't find any more bugs, but there still may be some bugs lurking. Hopefully there aren't though.

Also, if this ends up being merged, I plan to add support to SDL_CopyFile, SDL_CreateDirectory, SDL_RemovePath and SDL_RenamePath.

Existing Issue(s)

Fixes #15541.

[crudelios]

Regarding the cache, here's the approximate result of a full recursive directory traversal (with SDL_GetPathInfo for each file and a recursive SDL_EnumerateDirectory if a subdirectory is found), tested on a Samsung Galaxy S22 on a folder with 1356 subfiles and 27 subfolders:

• Before caching: ~3200 - 4500ms
• After caching: ~45-50ms

So it's quite a big difference.

Edit: Some more numbers:

The "Before caching" above wasn't entirely correct, as some retrievals were being cached during the initial enumeration.

Here are the most recent results:

• No cache: 30+ seconds
• Real URI's, cache enabled, first recursive traversal: ~3200-4500ms
• Real URI's, subsequent traversals: ~30-50ms
• SDL URI's, cache enabled, first recursive traversal: ~2700-4000ms
• SDL URI's, cache enabled, subsequent traversals: ~12-20ms

The difference is even more dramatic than I expected.