2020-01-01 21:29:24 +01:00
|
|
|
#ifndef FILESYSTEMUTILS_H
|
|
|
|
|
#define FILESYSTEMUTILS_H
|
|
|
|
|
|
Abstract binary blob loading to FileSystemUtils
This seems to be a comment left by Ethan that he never got around to. So
I did it for him.
What I've done is made it so FileSystemUtils.cpp knows what a binary
blob is, and moved the binary blob loading code directly to
FileSystemUtils.cpp. To do this, I removed the private access modifier
from binaryBlob - I don't think we'll need it, and anyways when we move
to C we can't use it.
Along the way, I also cleaned up the style of the function a bit - the
null termination offset is no longer hardcoded, and the function no
longer mixes code and declarations together in the same block.
I also noticed that when printing all the filenames at the end, a single
invalid header would stop the whole loop instead of just being skipped
over... this seems to be a bug to me, so I've made it so invalid headers
just get skipped over instead of stopping the whole loop.
In FileSystemUtils.h, I used a forward declaration. In hindsight,
incomplete forward declarations should basically always be done in
header files if possible, otherwise this introduces the possibility of
transitive includes - if a file includes this header and it does a full
include, the file is silently able to use the full header, whereas if
it's a forward declaration, then the moment the file tries to use the
full header it fails, and then it's forced to include the full header
for itself. But uh, that's a code cleanup for later.
2021-04-13 09:29:13 +02:00
|
|
|
/* Forward declaration */
|
|
|
|
|
class binaryBlob;
|
|
|
|
|
|
Refactor level dir listing to not use STL data marshalling
Note that level dir listing still uses plenty of STL (including the end
product - the `LevelMetaData` struct - which, for the purposes of 2.3,
is okay enough (2.4 should remove STL usage entirely)); it's just that
the initial act of iterating over the levels directory no longer takes
four or SIX(!!!) heap allocations (not counting reallocations and other
heap allocations this patch does not remove), and no longer does any
data marshalling.
Like text splitting, and binary blob extra indice grabbing, the current
approach that FILESYSTEM_getLevelDirFileNames() uses is a temporary
std::vector of std::strings as a middleman to store all the filenames,
and the game iterates over that std::vector to grab each level metadata.
Except, it's even worse in this case, because PHYSFS_enumerateFiles()
ALREADY does a heap allocation. Oh, and
FILESYSTEM_getLevelDirFileNames() gets called two or three times. Yeah,
let me explain:
1. FILESYSTEM_getLevelDirFileNames() calls PHYSFS_enumerateFiles().
2. PHYSFS_enumerateFiles() allocates an array of pointers to arrays of
chars on the heap. For each filename, it will:
a. Allocate an array of chars for the filename.
b. Reallocate the array of pointers to add the pointer to the above
char array.
(In this step, it also inserts the filename in alphabetically -
without any further allocations, as far as I know - but this is a
COMPLETELY unnecessary step, because we are going to sort the list
of levels by ourselves via the metadata title in the end anyways.)
3. FILESYSTEM_getLevelDirFileNames() iterates over the PhysFS list, and
allocates an std::vector on the heap to shove the list into. Then,
for each filename, it will:
a. Allocate an std::string, initialized to "levels/".
b. Append the filename to the std::string above. This will most
likely require a re-allocation.
c. Duplicate the std::string - which requires allocating more memory
again - to put it into the std::vector.
(Compared to the PhysFS list above, the std::vector does less
reallocations; it however will still end up reallocating a certain
amount of times in the end.)
4. FILESYSTEM_getLevelDirFileNames() will free the PhysFS list.
5. Then to get the std::vector<std::string> back to the caller, we end
up having to reallocate the std::vector again - reallocating every
single std::string inside it, too - to give it back to the caller.
And to top it all off, FILESYSTEM_getLevelDirFileNames() is guaranteed
to either be called two times, or three times. This is because
editorclass::getDirectoryData() will call editorclass::loadZips(), which
will unconditionally call FILESYSTEM_getLevelDirFileNames(), then call
it AGAIN if a zip was found. Then once the function returns,
getDirectoryData() will still unconditionally call
FILESYSTEM_getLevelDirFileNames(). This smells like someone bolting
something on without regard for the whole picture of the system, but
whatever; I can clean up their mess just fine.
So, what do I do about this? Well, just like I did with text splitting
and binary blob extras, make the final for-loop - the one that does the
actual metadata parsing - more immediate.
So how do I do that? Well, PhysFS has a function named
PHYSFS_enumerate(). PHYSFS_enumerateFiles(), in fact, uses this function
internally, and is basically just a wrapper with some allocation and
alphabetization.
PHYSFS_enumerate() takes in a pointer to a function, which it will call
for every single entry that it iterates over. It also lets you pass in
another arbitrary pointer that it leaves alone, which I use to pass
through a function pointer that is the actual callback.
So to clarify, there are two callbacks - one callback is passed through
into another callback that gets passed through to PHYSFS_enumerate().
The callback that gets passed to PHYSFS_enumerate() is always the same,
but the callback that gets passed through the callback can be different
(if you look at the calling code, you can see that one caller passes
through a normal level metadata callback; the other passes through a zip
file callback).
Furthermore, I've also cleaned it up so that if editorclass::loadZips()
finds a zip file, it won't iterate over all the files in the levels
directory a third time. Instead, the level directory only gets iterated
over twice - once to check for zips, and another to load every level
plus all zips; the second time is when all the heap allocations happen.
And with that, level list loading now uses less STL templated stuff and
much less heap allocations.
Also, ed.directoryList basically has no reason to exist other than being
a temporary std::vector, so I've removed it. This further decreases
memory usage, depending on how many levels you have in your levels
folder (I know that I usually have a lot and don't really ever clean it
up, lol).
Lastly, in the callback passed to PhysFS, `builtLocation` is actually no
longer hardcoded to just the `levels` directory, since instead we now
use the `origdir` variable that PhysFS passes us. So that's good, too.
2021-02-19 03:42:13 +01:00
|
|
|
#include <stddef.h>
|
2020-01-01 21:29:24 +01:00
|
|
|
|
2020-06-04 04:37:01 +02:00
|
|
|
// Forward declaration, including the entirety of tinyxml2.h across all files this file is included in is unnecessary
|
|
|
|
|
namespace tinyxml2 { class XMLDocument; }
|
Replace TiXmlDocument load and save functions by PHYSFS
The TinyXml functions to load and save files don't properly support
unicode file paths on Windows, so in order to support that properly, I
saw no other option than to do the actual loading and saving via PHYSFS
(or to use the Windows API on Windows and retain doc.LoadFile and
doc.SaveFile on other OSes, but that'd be more complicated and
unnecessary, we already have PHYSFS, right?).
There are two new functions in FileSystemUtils:
bool FILESYSTEM_saveTiXmlDocument(const char *name, TiXmlDocument *doc)
bool FILESYSTEM_loadTiXmlDocument(const char *name, TiXmlDocument *doc)
Any instances of doc.SaveFile(<FULL_PATH>) have been replaced by
FILESYSTEM_saveTiXmlDocument(<VVVVVV_FOLDER_PATH>, &doc), where
<FULL_PATH> included the full path to the saves or levels directory,
and <VVVVVV_FOLDER_PATH> only includes the path relative to the VVVVVV
directory.
When loading a document, a TiXmlDocument used to be created with a full
path in its constructor and doc.LoadFile() would then be called, now a
TiXmlDocument is constructed with no path name and
FILESYSTEM_loadTiXmlDocument(<VVVVVV_FOLDER_PATH>, &doc) is called.
2020-01-12 15:17:39 +01:00
|
|
|
|
2022-12-30 22:57:24 +01:00
|
|
|
int FILESYSTEM_init(char *argvZero, char* baseDir, char* assetsPath, char* langDir, char* fontsDir);
|
2022-03-14 18:45:19 +01:00
|
|
|
bool FILESYSTEM_isInit(void);
|
Explicitly declare void for all void parameter functions (#628)
Apparently in C, if you have `void test();`, it's completely okay to do
`test(2);`. The function will take in the argument, but just discard it
and throw it away. It's like a trash can, and a rude one at that. If you
declare it like `void test(void);`, this is prevented.
This is not a problem in C++ - doing `void test();` and `test(2);` is
guaranteed to result in a compile error (this also means that right now,
at least in all `.cpp` files, nobody is ever calling a void parameter
function with arguments and having their arguments be thrown away).
However, we may not be using C++ in the future, so I just want to lay
down the precedent that if a function takes in no arguments, you must
explicitly declare it as such.
I would've added `-Wstrict-prototypes`, but it produces an annoying
warning message saying it doesn't work in C++ mode if you're compiling
in C++ mode. So it can be added later.
2021-02-25 23:23:59 +01:00
|
|
|
void FILESYSTEM_deinit(void);
|
2020-01-01 21:29:24 +01:00
|
|
|
|
Explicitly declare void for all void parameter functions (#628)
Apparently in C, if you have `void test();`, it's completely okay to do
`test(2);`. The function will take in the argument, but just discard it
and throw it away. It's like a trash can, and a rude one at that. If you
declare it like `void test(void);`, this is prevented.
This is not a problem in C++ - doing `void test();` and `test(2);` is
guaranteed to result in a compile error (this also means that right now,
at least in all `.cpp` files, nobody is ever calling a void parameter
function with arguments and having their arguments be thrown away).
However, we may not be using C++ in the future, so I just want to lay
down the precedent that if a function takes in no arguments, you must
explicitly declare it as such.
I would've added `-Wstrict-prototypes`, but it produces an annoying
warning message saying it doesn't work in C++ mode if you're compiling
in C++ mode. So it can be added later.
2021-02-25 23:23:59 +01:00
|
|
|
char *FILESYSTEM_getUserSaveDirectory(void);
|
|
|
|
|
char *FILESYSTEM_getUserLevelDirectory(void);
|
2022-12-30 22:57:24 +01:00
|
|
|
char *FILESYSTEM_getUserMainLangDirectory(void);
|
|
|
|
|
bool FILESYSTEM_isMainLangDirFromRepo(void);
|
2023-08-30 16:45:04 +02:00
|
|
|
bool FILESYSTEM_doesLangDirExist(void);
|
|
|
|
|
bool FILESYSTEM_doesFontsDirExist(void);
|
2022-12-30 22:57:24 +01:00
|
|
|
|
|
|
|
|
bool FILESYSTEM_setLangWriteDir(void);
|
|
|
|
|
bool FILESYSTEM_restoreWriteDir(void);
|
2020-01-01 21:29:24 +01:00
|
|
|
|
2021-03-05 01:39:07 +01:00
|
|
|
bool FILESYSTEM_isFile(const char* filename);
|
2021-03-05 01:45:07 +01:00
|
|
|
bool FILESYSTEM_isMounted(const char* filename);
|
2021-03-05 01:39:07 +01:00
|
|
|
|
2021-03-05 01:05:06 +01:00
|
|
|
void FILESYSTEM_loadZip(const char* filename);
|
2021-08-07 05:57:34 +02:00
|
|
|
bool FILESYSTEM_mountAssets(const char *path);
|
2021-04-03 21:48:47 +02:00
|
|
|
void FILESYSTEM_unmountAssets(void);
|
2021-03-06 19:48:02 +01:00
|
|
|
bool FILESYSTEM_isAssetMounted(const char* filename);
|
2022-12-30 22:57:24 +01:00
|
|
|
bool FILESYSTEM_areAssetsInSameRealDir(const char* filenameA, const char* filenameB);
|
2020-06-01 01:31:02 +02:00
|
|
|
|
2024-01-09 20:29:50 +01:00
|
|
|
bool FILESYSTEM_saveFile(const char* name, const unsigned char* data, size_t len);
|
2020-01-24 21:35:46 +01:00
|
|
|
void FILESYSTEM_loadFileToMemory(const char *name, unsigned char **mem,
|
2023-03-18 23:12:24 +01:00
|
|
|
size_t *len);
|
2021-04-05 09:53:04 +02:00
|
|
|
void FILESYSTEM_loadAssetToMemory(
|
|
|
|
|
const char* name,
|
|
|
|
|
unsigned char** mem,
|
2023-03-18 23:12:24 +01:00
|
|
|
size_t* len
|
2021-04-05 09:53:04 +02:00
|
|
|
);
|
Abstract binary blob loading to FileSystemUtils
This seems to be a comment left by Ethan that he never got around to. So
I did it for him.
What I've done is made it so FileSystemUtils.cpp knows what a binary
blob is, and moved the binary blob loading code directly to
FileSystemUtils.cpp. To do this, I removed the private access modifier
from binaryBlob - I don't think we'll need it, and anyways when we move
to C we can't use it.
Along the way, I also cleaned up the style of the function a bit - the
null termination offset is no longer hardcoded, and the function no
longer mixes code and declarations together in the same block.
I also noticed that when printing all the filenames at the end, a single
invalid header would stop the whole loop instead of just being skipped
over... this seems to be a bug to me, so I've made it so invalid headers
just get skipped over instead of stopping the whole loop.
In FileSystemUtils.h, I used a forward declaration. In hindsight,
incomplete forward declarations should basically always be done in
header files if possible, otherwise this introduces the possibility of
transitive includes - if a file includes this header and it does a full
include, the file is silently able to use the full header, whereas if
it's a forward declaration, then the moment the file tries to use the
full header it fails, and then it's forced to include the full header
for itself. But uh, that's a code cleanup for later.
2021-04-13 09:29:13 +02:00
|
|
|
|
|
|
|
|
bool FILESYSTEM_loadBinaryBlob(binaryBlob* blob, const char* filename);
|
|
|
|
|
|
2021-09-02 19:19:51 +02:00
|
|
|
bool FILESYSTEM_saveTiXml2Document(const char *name, tinyxml2::XMLDocument& doc, bool sync = true);
|
2020-06-03 19:00:53 +02:00
|
|
|
bool FILESYSTEM_loadTiXml2Document(const char *name, tinyxml2::XMLDocument& doc);
|
2022-12-30 22:57:24 +01:00
|
|
|
bool FILESYSTEM_loadAssetTiXml2Document(const char *name, tinyxml2::XMLDocument& doc);
|
2020-01-01 21:29:24 +01:00
|
|
|
|
Refactor level dir listing to not use STL data marshalling
Note that level dir listing still uses plenty of STL (including the end
product - the `LevelMetaData` struct - which, for the purposes of 2.3,
is okay enough (2.4 should remove STL usage entirely)); it's just that
the initial act of iterating over the levels directory no longer takes
four or SIX(!!!) heap allocations (not counting reallocations and other
heap allocations this patch does not remove), and no longer does any
data marshalling.
Like text splitting, and binary blob extra indice grabbing, the current
approach that FILESYSTEM_getLevelDirFileNames() uses is a temporary
std::vector of std::strings as a middleman to store all the filenames,
and the game iterates over that std::vector to grab each level metadata.
Except, it's even worse in this case, because PHYSFS_enumerateFiles()
ALREADY does a heap allocation. Oh, and
FILESYSTEM_getLevelDirFileNames() gets called two or three times. Yeah,
let me explain:
1. FILESYSTEM_getLevelDirFileNames() calls PHYSFS_enumerateFiles().
2. PHYSFS_enumerateFiles() allocates an array of pointers to arrays of
chars on the heap. For each filename, it will:
a. Allocate an array of chars for the filename.
b. Reallocate the array of pointers to add the pointer to the above
char array.
(In this step, it also inserts the filename in alphabetically -
without any further allocations, as far as I know - but this is a
COMPLETELY unnecessary step, because we are going to sort the list
of levels by ourselves via the metadata title in the end anyways.)
3. FILESYSTEM_getLevelDirFileNames() iterates over the PhysFS list, and
allocates an std::vector on the heap to shove the list into. Then,
for each filename, it will:
a. Allocate an std::string, initialized to "levels/".
b. Append the filename to the std::string above. This will most
likely require a re-allocation.
c. Duplicate the std::string - which requires allocating more memory
again - to put it into the std::vector.
(Compared to the PhysFS list above, the std::vector does less
reallocations; it however will still end up reallocating a certain
amount of times in the end.)
4. FILESYSTEM_getLevelDirFileNames() will free the PhysFS list.
5. Then to get the std::vector<std::string> back to the caller, we end
up having to reallocate the std::vector again - reallocating every
single std::string inside it, too - to give it back to the caller.
And to top it all off, FILESYSTEM_getLevelDirFileNames() is guaranteed
to either be called two times, or three times. This is because
editorclass::getDirectoryData() will call editorclass::loadZips(), which
will unconditionally call FILESYSTEM_getLevelDirFileNames(), then call
it AGAIN if a zip was found. Then once the function returns,
getDirectoryData() will still unconditionally call
FILESYSTEM_getLevelDirFileNames(). This smells like someone bolting
something on without regard for the whole picture of the system, but
whatever; I can clean up their mess just fine.
So, what do I do about this? Well, just like I did with text splitting
and binary blob extras, make the final for-loop - the one that does the
actual metadata parsing - more immediate.
So how do I do that? Well, PhysFS has a function named
PHYSFS_enumerate(). PHYSFS_enumerateFiles(), in fact, uses this function
internally, and is basically just a wrapper with some allocation and
alphabetization.
PHYSFS_enumerate() takes in a pointer to a function, which it will call
for every single entry that it iterates over. It also lets you pass in
another arbitrary pointer that it leaves alone, which I use to pass
through a function pointer that is the actual callback.
So to clarify, there are two callbacks - one callback is passed through
into another callback that gets passed through to PHYSFS_enumerate().
The callback that gets passed to PHYSFS_enumerate() is always the same,
but the callback that gets passed through the callback can be different
(if you look at the calling code, you can see that one caller passes
through a normal level metadata callback; the other passes through a zip
file callback).
Furthermore, I've also cleaned it up so that if editorclass::loadZips()
finds a zip file, it won't iterate over all the files in the levels
directory a third time. Instead, the level directory only gets iterated
over twice - once to check for zips, and another to load every level
plus all zips; the second time is when all the heap allocations happen.
And with that, level list loading now uses less STL templated stuff and
much less heap allocations.
Also, ed.directoryList basically has no reason to exist other than being
a temporary std::vector, so I've removed it. This further decreases
memory usage, depending on how many levels you have in your levels
folder (I know that I usually have a lot and don't really ever clean it
up, lol).
Lastly, in the callback passed to PhysFS, `builtLocation` is actually no
longer hardcoded to just the `levels` directory, since instead we now
use the `origdir` variable that PhysFS passes us. So that's good, too.
2021-02-19 03:42:13 +01:00
|
|
|
void FILESYSTEM_enumerateLevelDirFileNames(void (*callback)(const char* filename));
|
2020-01-01 21:29:24 +01:00
|
|
|
|
2023-01-11 02:57:31 +01:00
|
|
|
struct EnumHandle
|
|
|
|
|
{
|
|
|
|
|
char** physfs_list;
|
|
|
|
|
char** _item;
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
const char* FILESYSTEM_enumerate(const char* folder, EnumHandle* handle);
|
|
|
|
|
const char* FILESYSTEM_enumerateAssets(const char* folder, EnumHandle* handle);
|
|
|
|
|
const char* FILESYSTEM_enumerateLanguageCodes(EnumHandle* handle);
|
|
|
|
|
void FILESYSTEM_freeEnumerate(EnumHandle* handle);
|
2022-12-30 22:57:24 +01:00
|
|
|
|
2021-08-07 07:26:48 +02:00
|
|
|
bool FILESYSTEM_levelDirHasError(void);
|
|
|
|
|
void FILESYSTEM_clearLevelDirError(void);
|
|
|
|
|
const char* FILESYSTEM_getLevelDirError(void);
|
2023-05-17 19:09:59 +02:00
|
|
|
void FILESYSTEM_setLevelDirError(const char* text, const char* args_index, ...);
|
2021-08-07 07:26:48 +02:00
|
|
|
|
Explicitly declare void for all void parameter functions (#628)
Apparently in C, if you have `void test();`, it's completely okay to do
`test(2);`. The function will take in the argument, but just discard it
and throw it away. It's like a trash can, and a rude one at that. If you
declare it like `void test(void);`, this is prevented.
This is not a problem in C++ - doing `void test();` and `test(2);` is
guaranteed to result in a compile error (this also means that right now,
at least in all `.cpp` files, nobody is ever calling a void parameter
function with arguments and having their arguments be thrown away).
However, we may not be using C++ in the future, so I just want to lay
down the precedent that if a function takes in no arguments, you must
explicitly declare it as such.
I would've added `-Wstrict-prototypes`, but it produces an annoying
warning message saying it doesn't work in C++ mode if you're compiling
in C++ mode. So it can be added later.
2021-02-25 23:23:59 +01:00
|
|
|
bool FILESYSTEM_openDirectoryEnabled(void);
|
2020-04-18 03:48:55 +02:00
|
|
|
bool FILESYSTEM_openDirectory(const char *dname);
|
|
|
|
|
|
2020-04-26 22:22:26 +02:00
|
|
|
bool FILESYSTEM_delete(const char *name);
|
2021-08-12 05:54:02 +02:00
|
|
|
void FILESYSTEM_deleteLevelSaves(void);
|
2020-04-26 22:22:26 +02:00
|
|
|
|
2020-01-01 21:29:24 +01:00
|
|
|
#endif /* FILESYSTEMUTILS_H */
|
