Scarfski/bx
1
0
Fork 0
mirror of https://github.com/bkaradzic/bx.git synced 2026-09-10 04:18:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

Added scanner. (#410)

Browse Source
This commit is contained in:
Branimir Karadžić
2026-07-25 12:23:58 -07:00
committed by GitHub
parent cd6720ce9a
commit 9916e720fc
21 changed files with 2157 additions and 1899 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
3rdparty/ini/README.md vendored
View File

@@ -1 +0,0 @@
https://github.com/mattiasgustavsson/libs/

1067
3rdparty/ini/ini.h vendored
View File

File diff suppressed because it is too large Load Diff

333
3rdparty/ini/ini.md vendored
View File

@@ -1,333 +0,0 @@
ini.h
=====
Library: [ini.h](../ini.h)
Examples
========
Loading an ini file and retrieving values
-----------------------------------------
```cpp
#define INI_IMPLEMENTATION
#include "ini.h"
#include <stdio.h>
#include <stdlib.h>
int main()
{
FILE* fp = fopen( "test.ini", "r" );
fseek( fp, 0, SEEK_END );
int size = ftell( fp );
fseek( fp, 0, SEEK_SET );
char* data = (char*) malloc( size + 1 );
fread( data, 1, size, fp );
data[ size ] = '\0';
fclose( fp );
ini_t* ini = ini_load( data );
free( data );
int second_index = ini_find_property( ini, INI_GLOBAL_SECTION, "SecondSetting" );
char const* second = ini_property_value( ini, INI_GLOBAL_SECTION, second_index );
printf( "%s=%s\n", "SecondSetting", second );
int section = ini_find_section( ini, "MySection" );
int third_index = ini_find_property( ini, section, "ThirdSetting" );
char const* third = ini_property_value( ini, section, third_index );
printf( "%s=%s\n", "ThirdSetting", third );
ini_destroy( ini );
return 0;
}
```
Creating a new ini file
-----------------------
```cpp
#define INI_IMPLEMENTATION
#include "ini.h"
#include <stdio.h>
#include <stdlib.h>
int main()
{
ini_t* ini = ini_create();
ini_property_add( ini, INI_GLOBAL_SECTION, "FirstSetting", "Test" );
ini_property_add( ini, INI_GLOBAL_SECTION, "SecondSetting", "2" );
int section = ini_section_add( ini, "MySection" );
ini_property_add( ini, section, "ThirdSetting", "Three" );
int size = ini_save( ini, NULL, 0 ); // Find the size needed
char* data = (char*) malloc( size );
size = ini_save( ini, data, size ); // Actually save the file
ini_destroy( ini );
FILE* fp = fopen( "test.ini", "w" );
fwrite( data, 1, size, fp );
fclose( fp );
free( data );
return 0;
}
```
API Documentation
=================
ini.h is a small library for reading classic .ini files. It is a single-header library, and does not need any .lib files
or other binaries, or any build scripts. To use it, you just include ini.h to get the API declarations. To get the
definitions, you must include ini.h from *one* single C or C++ file, and #define the symbol `INI_IMPLEMENTATION` before
you do.
Customization
-------------
There are a few different things in ini.h which are configurable by #defines. The customizations only affect the
implementation, so will only need to be defined in the file where you have the #define INI_IMPLEMENTATION.
Note that if all customizations are utilized, ini.h will include no external files whatsoever, which might be useful
if you need full control over what code is being built.
### Custom memory allocators
To store the internal data structures, ini.h needs to do dynamic allocation by calling `malloc`. Programs might want to
keep track of allocations done, or use custom defined pools to allocate memory from. ini.h allows for specifying custom
memory allocation functions for `malloc` and `free`.
This is done with the following code:
#define INI_IMPLEMENTATION
#define INI_MALLOC( ctx, size ) ( my_custom_malloc( ctx, size ) )
#define INI_FREE( ctx, ptr ) ( my_custom_free( ctx, ptr ) )
#include "ini.h"
where `my_custom_malloc` and `my_custom_free` are your own memory allocation/deallocation functions. The `ctx` parameter
is an optional parameter of type `void*`. When `ini_create` or `ini_load` is called, you can pass in a `memctx`
parameter, which can be a pointer to anything you like, and which will be passed through as the `ctx` parameter to every
`INI_MALLOC`/`INI_FREE` call. For example, if you are doing memory tracking, you can pass a pointer to your tracking
data as `memctx`, and in your custom allocation/deallocation function, you can cast the `ctx` param back to the
right type, and access the tracking data.
If no custom allocator is defined, ini.h will default to `malloc` and `free` from the C runtime library.
### Custom C runtime function
The library makes use of three additional functions from the C runtime library, and for full flexibility, it allows you
to substitute them for your own. Here's an example:
#define INI_IMPLEMENTATION
#define INI_MEMCPY( dst, src, cnt ) ( my_memcpy_func( dst, src, cnt ) )
#define INI_STRLEN( s ) ( my_strlen_func( s ) )
#define INI_STRICMP( s1, s2 ) ( my_stricmp_func( s1, s2 ) )
#include "ini.h"
If no custom function is defined, ini.h will default to the C runtime library equivalent.
ini_create
----------
ini_t* ini_create( void* memctx )
Instantiates a new, empty ini structure, which can be manipulated with other API calls, to fill it with data. To save it
out to an ini-file string, use `ini_save`. When no longer needed, it can be destroyed by calling `ini_destroy`.
`memctx` is a pointer to user defined data which will be passed through to the custom INI_MALLOC/INI_FREE calls. It can
be NULL if no user defined data is needed.
ini_load
--------
ini_t* ini_load( char const* data, void* memctx )
Parse the zero-terminated string `data` containing an ini-file, and create a new ini_t instance containing the data.
The instance can be manipulated with other API calls to enumerate sections/properties and retrieve values. When no
longer needed, it can be destroyed by calling `ini_destroy`. `memctx` is a pointer to user defined data which will be
passed through to the custom INI_MALLOC/INI_FREE calls. It can be NULL if no user defined data is needed.
ini_save
--------
int ini_save( ini_t const* ini, char* data, int size )
Saves an ini structure as a zero-terminated ini-file string, into the specified buffer. Returns the number of bytes
written, including the zero terminator. If `data` is NULL, nothing is written, but `ini_save` still returns the number
of bytes it would have written. If the size of `data`, as specified in the `size` parameter, is smaller than that
required, only part of the ini-file string will be written. `ini_save` still returns the number of bytes it would have
written had the buffer been large enough.
ini_destroy
-----------
void ini_destroy( ini_t* ini )
Destroy an `ini_t` instance created by calling `ini_load` or `ini_create`, releasing the memory allocated by it. No
further API calls are valid on an `ini_t` instance after calling `ini_destroy` on it.
ini_section_count
-----------------
int ini_section_count( ini_t const* ini )
Returns the number of sections in an ini file. There's at least one section in an ini file (the global section), but
there can be many more, each specified in the file by the section name wrapped in square brackets [ ].
ini_section_name
----------------
char const* ini_section_name( ini_t const* ini, int section )
Returns the name of the section with the specified index. `section` must be non-negative and less than the value
returned by `ini_section_count`, or `ini_section_name` will return NULL. The defined constant `INI_GLOBAL_SECTION` can
be used to indicate the global section.
ini_property_count
------------------
int ini_property_count( ini_t const* ini, int section )
Returns the number of properties belonging to the section with the specified index. `section` must be non-negative and
less than the value returned by `ini_section_count`, or `ini_section_name` will return 0. The defined constant
`INI_GLOBAL_SECTION` can be used to indicate the global section. Properties are declared in the ini-file on he format
`name=value`.
ini_property_name
-----------------
char const* ini_property_name( ini_t const* ini, int section, int property )
Returns the name of the property with the specified index `property` in the section with the specified index `section`.
`section` must be non-negative and less than the value returned by `ini_section_count`, and `property` must be
non-negative and less than the value returned by `ini_property_count`, or `ini_property_name` will return NULL. The
defined constant `INI_GLOBAL_SECTION` can be used to indicate the global section.
ini_property_value
------------------
char const* ini_property_value( ini_t const* ini, int section, int property )
Returns the value of the property with the specified index `property` in the section with the specified index `section`.
`section` must be non-negative and less than the value returned by `ini_section_count`, and `property` must be
non-negative and less than the value returned by `ini_property_count`, or `ini_property_value` will return NULL. The
defined constant `INI_GLOBAL_SECTION` can be used to indicate the global section.
ini_find_section
----------------
int ini_find_section( ini_t const* ini, char const* name, int name_length )
Finds the section with the specified name, and returns its index. `name_length` specifies the number of characters in
`name`, which does not have to be zero-terminated. If `name_length` is zero, the length is determined automatically, but
in this case `name` has to be zero-terminated. If no section with the specified name could be found, the value
`INI_NOT_FOUND` is returned.
ini_find_property
-----------------
int ini_find_property( ini_t const* ini, int section, char const* name, int name_length )
Finds the property with the specified name, within the section with the specified index, and returns the index of the
property. `name_length` specifies the number of characters in `name`, which does not have to be zero-terminated. If
`name_length` is zero, the length is determined automatically, but in this case `name` has to be zero-terminated. If no
property with the specified name could be found within the specified section, the value `INI_NOT_FOUND` is returned.
`section` must be non-negative and less than the value returned by `ini_section_count`, or `ini_find_property` will
return `INI_NOT_FOUND`. The defined constant `INI_GLOBAL_SECTION` can be used to indicate the global section.
ini_section_add
---------------
int ini_section_add( ini_t* ini, char const* name, int length )
Adds a section with the specified name, and returns the index it was added at. There is no check done to see if a
section with the specified name already exists - multiple sections of the same name are allowed. `length` specifies the
number of characters in `name`, which does not have to be zero-terminated. If `length` is zero, the length is determined
automatically, but in this case `name` has to be zero-terminated.
ini_property_add
----------------
void ini_property_add( ini_t* ini, int section, char const* name, int name_length, char const* value, int value_length )
Adds a property with the specified name and value to the specified section, and returns the index it was added at. There
is no check done to see if a property with the specified name already exists - multiple properties of the same name are
allowed. `name_length` and `value_length` specifies the number of characters in `name` and `value`, which does not have
to be zero-terminated. If `name_length` or `value_length` is zero, the length is determined automatically, but in this
case `name`/`value` has to be zero-terminated. `section` must be non-negative and less than the value returned by
`ini_section_count`, or the property will not be added. The defined constant `INI_GLOBAL_SECTION` can be used to
indicate the global section.
ini_section_remove
------------------
void ini_section_remove( ini_t* ini, int section )
Removes the section with the specified index, and all properties within it. `section` must be non-negative and less than
the value returned by `ini_section_count`. The defined constant `INI_GLOBAL_SECTION` can be used to indicate the global
section. Note that removing a section will shuffle section indices, so that section indices you may have stored will no
longer indicate the same section as it did before the remove. Use the find functions to update your indices.
ini_property_remove
-------------------
void ini_property_remove( ini_t* ini, int section, int property )
Removes the property with the specified index from the specified section. `section` must be non-negative and less than
the value returned by `ini_section_count`, and `property` must be non-negative and less than the value returned by
`ini_property_count`. The defined constant `INI_GLOBAL_SECTION` can be used to indicate the global section. Note that
removing a property will shuffle property indices within the specified section, so that property indices you may have
stored will no longer indicate the same property as it did before the remove. Use the find functions to update your
indices.
ini_section_name_set
--------------------
void ini_section_name_set( ini_t* ini, int section, char const* name, int length )
Change the name of the section with the specified index. `section` must be non-negative and less than the value returned
by `ini_section_count`. The defined constant `INI_GLOBAL_SECTION` can be used to indicate the global section. `length`
specifies the number of characters in `name`, which does not have to be zero-terminated. If `length` is zero, the length
is determined automatically, but in this case `name` has to be zero-terminated.
ini_property_name_set
---------------------
void ini_property_name_set( ini_t* ini, int section, int property, char const* name, int length )
Change the name of the property with the specified index in the specified section. `section` must be non-negative and
less than the value returned by `ini_section_count`, and `property` must be non-negative and less than the value
returned by `ini_property_count`. The defined constant `INI_GLOBAL_SECTION` can be used to indicate the global section.
`length` specifies the number of characters in `name`, which does not have to be zero-terminated. If `length` is zero,
the length is determined automatically, but in this case `name` has to be zero-terminated.
ini_property_value_set
----------------------
void ini_property_value_set( ini_t* ini, int section, int property, char const* value, int length )
Change the value of the property with the specified index in the specified section. `section` must be non-negative and
less than the value returned by `ini_section_count`, and `property` must be non-negative and less than the value
returned by `ini_property_count`. The defined constant `INI_GLOBAL_SECTION` can be used to indicate the global section.
`length` specifies the number of characters in `value`, which does not have to be zero-terminated. If `length` is zero,
the length is determined automatically, but in this case `value` has to be zero-terminated.

228
include/bx/inline/scanner.inl Normal file
View File

@@ -0,0 +1,228 @@
/*
* Copyright 2010-2026 Branimir Karadzic. All rights reserved.
* License: https://github.com/bkaradzic/bx/blob/master/LICENSE
*/
#ifndef BX_SCANNER_H_HEADER_GUARD
# error "Must be included from bx/scanner.h!"
#endif // BX_SCANNER_H_HEADER_GUARD
namespace bx
{
inline Scanner::Scanner(const StringView& _input)
: m_input(_input)
, m_tail(_input)
, m_line(0)
{
}
inline void Scanner::reset()
{
m_tail = m_input;
m_line = 0;
}
inline StringView Scanner::acceptAll()
{
StringView result = m_tail;
moveBy(result.getLength() );
return result;
}
inline StringView Scanner::accept()
{
return moveBy(1);
}
inline StringView Scanner::accept(char _ch)
{
if (!m_tail.isEmpty()
&& _ch == *m_tail.getPtr() )
{
return moveBy(1);
}
return StringView();
}
inline StringView Scanner::accept(char _ch0, char _ch1)
{
if (!m_tail.isEmpty() )
{
const char ch = *m_tail.getPtr();
if (ch == _ch0
|| ch == _ch1)
{
return moveBy(1);
}
}
return StringView();
}
template<typename ...Args>
inline StringView Scanner::accept(char _ch0, char _ch1, Args... _args)
{
StringView result = accept(_ch0, _ch1);
if (!result.isEmpty() )
{
return result;
}
return accept(_args...);
}
inline StringView Scanner::accept(const StringView& _str)
{
if (hasPrefix(m_tail, _str) )
{
return moveTo({ m_tail.getPtr(), _str.getLength() });
}
return StringView();
}
inline StringView Scanner::accept(Class _class)
{
return moveTo({ m_tail.getPtr(), strFunc(_class).getPtr() });
}
inline StringView Scanner::accept(CharTestFn _fn)
{
if (!m_tail.isEmpty()
&& _fn(*m_tail.getPtr() ) )
{
return moveBy(1);
}
return StringView();
}
inline StringView Scanner::acceptWhile(const StringView& _any)
{
const StringView input = strLTrim(m_tail, _any);
if (input.getPtr() != m_tail.getPtr() )
{
return moveTo({ m_tail.getPtr(), input.getPtr() });
}
return StringView();
}
inline StringView Scanner::acceptWhile(CharTestFn _fn)
{
const char* ptr = m_tail.getPtr();
const char* term = m_tail.getTerm();
while (ptr != term
&& _fn(*ptr) )
{
++ptr;
}
return moveTo({ m_tail.getPtr(), ptr });
}
inline StringView Scanner::acceptUntil(const StringView& _find)
{
StringView result = strFind(m_tail, _find);
if (!result.isEmpty() )
{
return moveTo({ m_tail.getPtr(), result.getPtr() });
}
return result;
}
inline StringView Scanner::acceptUntil(Class _class)
{
StringView result = strFunc(_class);
return moveTo({ m_tail.getPtr(), result.getPtr() });
}
inline StringView Scanner::peek() const
{
return m_tail.isEmpty()
? StringView()
: StringView(m_tail.getPtr(), m_tail.getPtr() + 1)
;
}
inline StringView Scanner::peek(char _ch) const
{
if (!m_tail.isEmpty()
&& _ch == *m_tail.getPtr() )
{
return StringView(m_tail.getPtr(), m_tail.getPtr() + 1);
}
return StringView();
}
inline StringView Scanner::peek(const StringView& _str) const
{
if (hasPrefix(m_tail, _str) )
{
return StringView(m_tail.getPtr(), _str.getLength() );
}
return StringView();
}
inline StringView Scanner::peek(Class _class) const
{
return StringView(m_tail.getPtr(), strFunc(_class).getPtr() );
}
inline bool Scanner::seek(int32_t _bytes)
{
moveBy(_bytes);
return true;
}
inline uint32_t Scanner::getLine() const
{
return m_line;
}
inline StringView Scanner::between(const StringView& _from) const
{
const char* a = _from.getPtr();
const char* b = m_tail.getPtr();
return a <= b ? StringView(a, b) : StringView(b, a);
}
inline bool Scanner::isDone() const
{
return m_tail.isEmpty();
}
inline LineReader::LineReader(const StringView& _str)
: m_scanner(_str)
, m_line(0)
{
}
inline void LineReader::reset()
{
m_scanner.reset();
m_line = 0;
}
inline uint32_t LineReader::getLine() const
{
return m_line;
}
inline bool LineReader::isDone() const
{
return m_scanner.isDone();
}
} // namespace bx

149
include/bx/inline/string.inl
View File

@@ -461,50 +461,121 @@ namespace bx
return StringView(m_ptr, m_len);
}
inline BX_CONSTEXPR_FUNC bool isInRange(char _ch, char _from, char _to)
{
return unsigned(_ch - _from) <= unsigned(_to-_from);
}
inline BX_CONSTEXPR_FUNC bool isSpace(char _ch)
{
return ' ' == _ch // Space.
|| '\t' == _ch // Horizontal tab.
|| '\n' == _ch // Line feed / new line.
|| '\r' == _ch // Carriage return.
|| '\v' == _ch // Vertical tab.
|| '\f' == _ch // Form feed / new page.
;
}
inline BX_CONSTEXPR_FUNC bool isSpaceHoriz(char _ch)
{
return false
|| ' ' == _ch
|| '\t' == _ch
|| '\r' == _ch
;
}
inline BX_CONSTEXPR_FUNC bool isUpper(char _ch)
{
return isInRange(_ch, 'A', 'Z');
}
inline BX_CONSTEXPR_FUNC char toUpper(char _ch)
{
return _ch - (isLower(_ch) ? 0x20 : 0);
}
inline BX_CONSTEXPR_FUNC bool isLower(char _ch)
{
return isInRange(_ch, 'a', 'z');
}
inline BX_CONSTEXPR_FUNC char toLower(char _ch)
{
return _ch + (isUpper(_ch) ? 0x20 : 0);
}
inline BX_CONSTEXPR_FUNC bool isAlpha(char _ch)
{
return isLower(_ch) || isUpper(_ch);
}
inline BX_CONSTEXPR_FUNC bool isNumeric(char _ch)
{
return isInRange(_ch, '0', '9');
}
inline BX_CONSTEXPR_FUNC bool isAlphaNum(char _ch)
{
return false
|| isAlpha(_ch)
|| isNumeric(_ch)
;
}
inline BX_CONSTEXPR_FUNC bool isHexNum(char _ch)
{
return false
|| isInRange(toLower(_ch), 'a', 'f')
|| isNumeric(_ch)
;
}
inline BX_CONSTEXPR_FUNC bool isOctNum(char _ch)
{
return isInRange(_ch, '0', '7');
}
inline BX_CONSTEXPR_FUNC bool isBinNum(char _ch)
{
return isInRange(_ch, '0', '1');
}
inline BX_CONSTEXPR_FUNC bool isPrint(char _ch)
{
return isInRange(_ch, ' ', '~');
}
inline BX_CONSTEXPR_FUNC bool isIdentStart(char _ch)
{
return false
|| isAlpha(_ch)
|| '_' == _ch
;
}
inline BX_CONSTEXPR_FUNC bool isIdentChar(char _ch)
{
return false
|| isAlphaNum(_ch)
|| '_' == _ch
;
}
inline BX_CONSTEXPR_FUNC bool isPathSeparator(char _ch)
{
return false
|| '/' == _ch
|| '\\' == _ch
;
}
inline StringView strSubstr(const StringView& _str, int32_t _start, int32_t _len)
{
return StringView(_str, _start, _len);
}
inline LineReader::LineReader(const StringView& _str)
: m_str(_str)
{
reset();
}
inline void LineReader::reset()
{
m_curr = m_str;
m_line = 0;
}
inline StringView LineReader::next()
{
if (m_curr.getPtr() != m_str.getTerm() )
{
++m_line;
StringView curr(m_curr);
m_curr = strFindNl(m_curr);
StringView line(curr.getPtr(), m_curr.getPtr() );
return strRTrim(strRTrim(line, "\n"), "\r");
}
return m_curr;
}
inline bool LineReader::isDone() const
{
return m_curr.getPtr() == m_str.getTerm();
}
inline uint32_t LineReader::getLine() const
{
return m_line;
}
inline constexpr int32_t strLen(const StringView& _str, int32_t _max)
{
return min(_str.getLength(), _max);

341
include/bx/scanner.h Normal file
View File

@@ -0,0 +1,341 @@
/*
* Copyright 2011-2026 Branimir Karadzic. All rights reserved.
* License: https://github.com/bkaradzic/bx/blob/master/LICENSE
*/
#ifndef BX_SCANNER_H_HEADER_GUARD
#define BX_SCANNER_H_HEADER_GUARD
#include <bx/string.h>
namespace bx
{
/// Forward-only cursor over a string view, used to tokenize text input.
///
/// Scanner keeps a cursor inside immutable input string, and tracks line number as cursor
/// moves. `accept*` functions advance cursor when they match, `peek*` functions perform the
/// same test without moving cursor.
///
/// All returned string views point into input string, and no copies are made.
///
/// @attention Scanner doesn't own input string. Input string must outlive Scanner.
///
class Scanner
{
public:
/// Character class matched by `accept`, `peek`, and `acceptUntil`.
///
enum class Class : uint8_t
{
Space, //!< Run of whitespace characters.
NonSpace, //!< Run of non-whitespace characters.
Identifier, //!< Run of alphanumeric characters and `_`.
EndOfLine, //!< Everything up to, but not including, line terminator.
NewLine, //!< Everything up to, and including, line terminator.
};
/// Position inside input string returned by `getCursor`.
///
enum class Cursor : uint8_t
{
Current, //!< Current cursor position.
LineStart, //!< First character of line cursor is on.
LineEnd, //!< Position of `\n` that terminates line cursor is on, or end of input.
};
/// Default constructor is not available. Scanner must be constructed with input string.
///
Scanner() = delete;
/// Constructor.
///
/// @param[in] _input Input string to scan. It's not copied, and it must outlive Scanner.
///
Scanner(const StringView& _input);
/// Rewind cursor to the beginning of input string, and reset line counter.
///
void reset();
/// Accept everything from cursor to the end of input string.
///
/// @returns Accepted string view, or empty string view if input is exhausted.
///
StringView acceptAll();
/// Accept single character.
///
/// @returns Accepted character, or empty string view if input is exhausted.
///
StringView accept();
/// Accept single character if it matches `_ch`.
///
/// @param[in] _ch Character to match.
///
/// @returns Accepted character, or empty string view if it doesn't match.
///
StringView accept(char _ch);
/// Accept single character if it matches any of the characters passed in.
///
/// @param[in] _ch0 First character to match.
/// @param[in] _ch1 Second character to match.
///
/// @returns Accepted character, or empty string view if it doesn't match.
///
StringView accept(char _ch0, char _ch1);
/// Accept single character if it matches any of the characters passed in.
///
/// @param[in] _ch0 First character to match.
/// @param[in] _ch1 Second character to match.
/// @param[in] _args Additional characters to match.
///
/// @returns Accepted character, or empty string view if it doesn't match.
///
template<typename ...Args>
StringView accept(char _ch0, char _ch1, Args... _args);
/// Accept `_str` if input at cursor starts with it.
///
/// @param[in] _str String to match.
///
/// @returns Accepted string view, or empty string view if it doesn't match.
///
StringView accept(const StringView& _str);
/// Accept leading run of characters belonging to `_class`.
///
/// @param[in] _class Character class to match.
///
/// @returns Accepted string view, or empty string view if it doesn't match.
///
StringView accept(Class _class);
/// Accept single character if `_fn` returns true for it.
///
/// @param[in] _fn Character test function.
///
/// @returns Accepted character, or empty string view if it doesn't match.
///
StringView accept(CharTestFn _fn);
/// Accept characters as long as they are contained in `_any`.
///
/// @param[in] _any Set of characters to match.
///
/// @returns Accepted string view, or empty string view if it doesn't match.
///
StringView acceptWhile(const StringView& _any);
/// Accept characters as long as `_fn` returns true for them.
///
/// @param[in] _fn Character test function.
///
/// @returns Accepted string view, or empty string view if it doesn't match.
///
StringView acceptWhile(CharTestFn _fn);
/// Accept characters until `_find` is found. Cursor stops at `_find`, and `_find` is not
/// accepted.
///
/// @param[in] _find String to search for.
///
/// @returns Accepted string view, or empty string view if `_find` is not found, or it's
/// already at cursor.
///
/// @attention Returned string view is empty in both cases, and cursor doesn't move.
/// Use `getCursor` and `between` to capture text that can legitimately be empty.
///
StringView acceptUntil(const StringView& _find);
/// Accept leading run of characters belonging to `_class`.
///
/// @param[in] _class Character class to match.
///
/// @returns Accepted string view, or empty string view if it doesn't match.
///
/// @remarks Same as `accept(Class)`. It's provided for readability when used with
/// `Class::EndOfLine` and `Class::NewLine`.
///
StringView acceptUntil(Class _class);
/// Peek at single character without moving cursor.
///
/// @returns Next character, or empty string view if input is exhausted.
///
StringView peek() const;
/// Peek at single character without moving cursor, if it matches `_ch`.
///
/// @param[in] _ch Character to match.
///
/// @returns Next character, or empty string view if it doesn't match.
///
StringView peek(char _ch) const;
/// Peek at `_str` without moving cursor, if input at cursor starts with it.
///
/// @param[in] _str String to match.
///
/// @returns Matched string view, or empty string view if it doesn't match.
///
StringView peek(const StringView& _str) const;
/// Peek at leading run of characters belonging to `_class`, without moving cursor.
///
/// @param[in] _class Character class to match.
///
/// @returns Matched string view. It's zero length if it doesn't match.
///
StringView peek(Class _class) const;
/// Move cursor to `_to`. Line number is adjusted accordingly, and cursor can move backward.
///
/// @param[in] _to Position to move cursor to. It must be a string view into input string.
///
/// @returns True if cursor moved, or false if `_to` is not a string view into input string.
///
bool seek(const StringView& _to);
/// Move cursor by `_bytes`. Negative value moves cursor backward. Line number is adjusted
/// accordingly, and value is clamped to input string bounds.
///
/// @param[in] _bytes Number of bytes to move cursor by.
///
/// @returns Always true.
///
bool seek(int32_t _bytes);
/// Returns zero-based line number cursor is on.
///
/// @returns Line number.
///
uint32_t getLine() const;
/// Returns one-based column number cursor is on.
///
/// @returns Column number.
///
uint32_t getColumn() const;
/// Returns cursor position as zero length string view.
///
/// @param[in] _which Position to return.
///
/// @returns Zero length string view at requested position.
///
/// @remarks Unlike default constructed `StringView`, returned string view keeps pointer
/// into input string even though it's empty. Use it with `between` to capture text that
/// can legitimately be empty.
///
StringView getCursor(Cursor _which = Cursor::Current) const;
/// Returns text between `_from` and current cursor position.
///
/// @param[in] _from Position to measure from, usually obtained by `getCursor`.
///
/// @returns String view between `_from` and cursor. Order of arguments doesn't matter,
/// `_from` can be either before or after cursor.
///
StringView between(const StringView& _from) const;
/// Returns true if input string is exhausted.
///
/// @returns True if cursor reached end of input string, otherwise returns false.
///
bool isDone() const;
private:
/// Move cursor to `_to`, and update line number.
///
/// @param[in] _to Position to move cursor to.
///
/// @returns String view between old and new cursor position, or empty string view if `_to`
/// is empty.
///
StringView moveTo(const StringView& _to);
/// Move cursor by `_bytes` clamped to input string bounds, and update line number.
///
/// @param[in] _bytes Number of bytes to move cursor by.
///
/// @returns String view between old and new cursor position.
///
StringView moveBy(int32_t _bytes);
/// Returns end of leading run of characters belonging to `_class`.
///
/// @param[in] _class Character class to match.
///
/// @returns String view starting past the run, and ending at end of input string.
///
StringView strFunc(Class _class) const;
/// Count number of line terminators in `_str`.
///
/// @param[in] _str String to count line terminators in.
///
/// @returns Number of lines.
///
static uint32_t countLines(const StringView& _str);
const StringView m_input;
StringView m_tail;
uint32_t m_line;
};
/// Splits string into lines.
///
/// Line terminator is `\n` or `\r\n`, and it's not part of returned line. Trailing `\r` left
/// by malformed input is trimmed.
///
/// All returned string views point into input string, and no copies are made.
///
/// @attention LineReader doesn't own input string. Input string must outlive LineReader.
///
class LineReader
{
public:
/// Constructor.
///
/// @param[in] _str Input string to read lines from. It's not copied, and it must outlive
/// LineReader.
///
LineReader(const StringView& _str);
/// Rewind to the beginning of input string, and reset line counter.
///
void reset();
/// Read next line.
///
/// @returns Next line without line terminator. Line can be empty, and returned string view
/// keeps pointer into input string. If input is exhausted, zero length string view at
/// end of input string is returned.
///
StringView next();
/// Returns one-based line number of line returned by the last `next` call.
///
/// @returns Line number, or 0 if `next` was not called yet.
///
uint32_t getLine() const;
/// Returns true if input string is exhausted.
///
/// @returns True if all lines are read, otherwise returns false.
///
bool isDone() const;
private:
Scanner m_scanner;
uint32_t m_line;
};
} // namespace bx
#include "inline/scanner.inl"
#endif // BX_SCANNER_H_HEADER_GUARD

4
include/bx/settings.h
View File

@@ -46,8 +46,10 @@ namespace bx
private:
Settings();
struct Ini;
AllocatorI* m_allocator;
void* m_ini;
Ini* m_ini;
};
///

65
include/bx/string.h
View File

@@ -279,6 +279,8 @@ namespace bx
int32_t m_capacity;
};
typedef bool (*CharTestFn)(char _ch);
/// Returns true if character is part of white space set.
///
/// White space set is:
@@ -289,55 +291,73 @@ namespace bx
/// '\v' - Vertical tab.
/// '\f' - Form feed / new page.
///
bool isSpace(char _ch);
BX_CONSTEXPR_FUNC bool isSpace(char _ch);
/// Returns true if string view contains only space characters.
bool isSpace(const StringView& _str);
/// Returns true if character is horizontal white space.
BX_CONSTEXPR_FUNC bool isSpaceHoriz(char _ch);
/// Returns true if character is uppercase.
bool isUpper(char _ch);
BX_CONSTEXPR_FUNC bool isUpper(char _ch);
/// Returns true if string view contains only uppercase characters.
bool isUpper(const StringView& _str);
/// Returns true if character is lowercase.
bool isLower(char _ch);
BX_CONSTEXPR_FUNC bool isLower(char _ch);
/// Returns true if string view contains only lowercase characters.
bool isLower(const StringView& _str);
/// Returns true if character is part of alphabet set.
bool isAlpha(char _ch);
BX_CONSTEXPR_FUNC bool isAlpha(char _ch);
/// Returns true if string view contains only alphabet characters.
bool isAlpha(const StringView& _str);
/// Returns true if character is part of numeric set.
bool isNumeric(char _ch);
BX_CONSTEXPR_FUNC bool isNumeric(char _ch);
/// Returns true if string view contains only numeric characters.
bool isNumeric(const StringView& _str);
/// Returns true if character is part of alpha numeric set.
bool isAlphaNum(char _ch);
BX_CONSTEXPR_FUNC bool isAlphaNum(char _ch);
/// Returns true if string view contains only alphanumeric characters.
bool isAlphaNum(const StringView& _str);
/// Returns true if character is part of hexadecimal set.
bool isHexNum(char _ch);
BX_CONSTEXPR_FUNC bool isHexNum(char _ch);
/// Returns true if string view contains only hexadecimal characters.
bool isHexNum(const StringView& _str);
///
BX_CONSTEXPR_FUNC bool isOctNum(char _ch);
///
BX_CONSTEXPR_FUNC bool isBinNum(char _ch);
///
BX_CONSTEXPR_FUNC bool isIdentStart(char _ch);
///
BX_CONSTEXPR_FUNC bool isIdentChar(char _ch);
///
BX_CONSTEXPR_FUNC bool isPathSeparator(char _ch);
/// Returns true if character is printable.
bool isPrint(char _ch);
BX_CONSTEXPR_FUNC bool isPrint(char _ch);
/// Returns true if string vieww contains only printable characters.
bool isPrint(const StringView& _str);
/// Returns lower case character representing _ch.
char toLower(char _ch);
BX_CONSTEXPR_FUNC char toLower(char _ch);
/// Lower case string in place assuming length passed is valid.
void toLowerUnsafe(char* _inOutStr, int32_t _len);
@@ -346,7 +366,7 @@ namespace bx
void toLower(char* _inOutStr, int32_t _max = INT32_MAX);
/// Returns upper case character representing _ch.
char toUpper(char _ch);
BX_CONSTEXPR_FUNC char toUpper(char _ch);
/// Upper case string in place assuming length passed is valid.
void toUpperUnsafe(char* _inOutStr, int32_t _len);
@@ -557,31 +577,6 @@ namespace bx
/// Converts string to 64-bit unsigned long long value.
bool fromString(unsigned long long* _out, const StringView& _str);
///
class LineReader
{
public:
///
LineReader(const StringView& _str);
///
void reset();
///
StringView next();
///
bool isDone() const;
///
uint32_t getLine() const;
private:
const StringView m_str;
StringView m_curr;
uint32_t m_line;
};
} // namespace bx
#include "inline/string.inl"

1
scripts/bx.lua
View File

@@ -78,6 +78,7 @@ project "bx"
path.join(BX_DIR, "src/mutex.cpp"),
path.join(BX_DIR, "src/os.cpp"),
path.join(BX_DIR, "src/process.cpp"),
path.join(BX_DIR, "src/scanner.cpp"),
path.join(BX_DIR, "src/semaphore.cpp"),
path.join(BX_DIR, "src/settings.cpp"),
path.join(BX_DIR, "src/sort.cpp"),

1
src/amalgamated.cpp
View File

@@ -18,6 +18,7 @@
#include "mutex.cpp"
#include "os.cpp"
#include "process.cpp"
#include "scanner.cpp"
#include "semaphore.cpp"
#include "settings.cpp"
#include "sort.cpp"

29
src/debug.cpp
View File

@@ -8,6 +8,7 @@
#include <bx/readerwriter.h> // WriterI
#include <bx/os.h> // exit
#include <bx/process.h> // ProcessReader
#include <bx/scanner.h> // Scanner
#include <inttypes.h> // PRIx*
@@ -947,22 +948,6 @@ namespace bx
#elif BX_CONFIG_CALLSTACK_USE_EXECINFO
StringView strConsumeTo(StringView& _input, const StringView& _find)
{
const StringView to = strFind(_input, _find);
if (!to.isEmpty() )
{
const StringView result(_input.getPtr(), to.getPtr() );
_input.set(to.getTerm(), _input.getTerm() );
return result;
}
return StringView();
}
int32_t writeCallstack(WriterI* _writer, const uintptr_t* _stack, uint32_t _num, Error* _err)
{
int32_t total = write(_writer, _err, "Callstack (%d):\n", _num);
@@ -1126,16 +1111,18 @@ namespace bx
{
for (LineReader lr({atosBuffer, bytes}); !lr.isDone();)
{
StringView input = lr.next();
Scanner scanner(lr.next() );
atosFunctionName = strConsumeTo(input, " (");
atosFunctionName = scanner.acceptUntil(" (");
if (atosFunctionName.isEmpty() )
{
break;
}
filePath = strConsumeTo(input, ":");
scanner.accept(" (");
filePath = scanner.acceptUntil(":");
if (filePath.isEmpty() )
{
@@ -1143,7 +1130,9 @@ namespace bx
break;
}
const StringView lineStr = strConsumeTo(input, ")");
scanner.accept(':');
const StringView lineStr = scanner.acceptUntil(")");
if (!lineStr.isEmpty() )
{

135
src/filepath.cpp
View File

@@ -6,6 +6,7 @@
#include <bx/file.h>
#include <bx/os.h>
#include <bx/readerwriter.h>
#include <bx/scanner.h>
#if BX_CRT_MSVC
# include <direct.h> // _getcwd
@@ -24,12 +25,9 @@ extern "C" int _NSGetExecutablePath(char* _buf, uint32_t* _bufSize);
namespace bx
{
static bool isPathSeparator(char _ch)
static bool isNotPathSeparator(char _ch)
{
return false
|| '/' == _ch
|| '\\' == _ch
;
return !isPathSeparator(_ch);
}
static int32_t normalizeFilePath(char* _dst, int32_t _dstSize, const char* _src, int32_t _num)
@@ -38,9 +36,9 @@ namespace bx
// - Lexical File Names in Plan 9 or Getting Dot-Dot Right
// https://web.archive.org/web/20180629044444/https://9p.io/sys/doc/lexnames.html
const int32_t num = strLen(_src, _num);
const StringView src(_src, strLen(_src, _num) );
if (0 == num)
if (src.isEmpty() )
{
return strCopy(_dst, _dstSize, ".");
}
@@ -50,91 +48,80 @@ namespace bx
StaticMemoryBlockWriter writer(_dst, _dstSize);
Error err;
int32_t idx = 0;
int32_t dotdot = 0;
Scanner scanner(src);
if (2 <= num
&& ':' == _src[1])
// Everything below `dotdot` is a prefix `..` can't back out of.
int32_t dotdot = 0;
if (2 <= src.getLength()
&& ':' == src.getPtr()[1])
{
size += write(&writer, toUpper(_src[idx]), &err);
size += write(&writer, toUpper(src.getPtr()[0]), &err);
size += write(&writer, ':', &err);
idx += 2;
scanner.seek(2);
dotdot = size;
}
const int32_t slashIdx = idx;
const bool rooted = !scanner.accept(isPathSeparator).isEmpty();
bool rooted = isPathSeparator(_src[idx]);
if (rooted)
{
size += write(&writer, '/', &err);
++idx;
dotdot = size;
}
const int32_t rootSize = size;
bool trailingSlash = false;
while (idx < num && err.isOk() )
while (!scanner.isDone()
&& err.isOk() )
{
switch (_src[idx])
if (!scanner.acceptWhile(isPathSeparator).isEmpty() )
{
case '/':
case '\\':
++idx;
trailingSlash = idx == num;
break;
case '.':
if (idx+1 == num
|| isPathSeparator(_src[idx+1]) )
{
++idx;
break;
}
if ('.' == _src[idx+1]
&& (idx+2 == num || isPathSeparator(_src[idx+2]) ) )
{
idx += 2;
if (dotdot < size)
{
for (--size
; dotdot < size && !isPathSeparator(_dst[size])
; --size)
{
}
seek(&writer, size, Whence::Begin);
}
else if (!rooted)
{
if (0 < size)
{
size += write(&writer, '/', &err);
}
size += write(&writer, "..", &err);
dotdot = size;
}
break;
}
[[fallthrough]];
default:
if ( ( rooted && slashIdx+1 != size)
|| (!rooted && 0 != size) )
{
size += write(&writer, '/', &err);
}
for (; idx < num && !isPathSeparator(_src[idx]); ++idx)
{
size += write(&writer, _src[idx], &err);
}
break;
trailingSlash = scanner.isDone();
continue;
}
const StringView component = scanner.acceptWhile(isNotPathSeparator);
if (component == ".")
{
continue;
}
if (component == "..")
{
if (dotdot < size)
{
for (--size
; dotdot < size && !isPathSeparator(_dst[size])
; --size)
{
}
seek(&writer, size, Whence::Begin);
}
else if (!rooted)
{
if (0 < size)
{
size += write(&writer, '/', &err);
}
size += write(&writer, "..", &err);
dotdot = size;
}
continue;
}
if (rooted ? rootSize != size : 0 != size)
{
size += write(&writer, '/', &err);
}
size += write(&writer, component, &err);
}
if (0 == size)

167
src/scanner.cpp Normal file
View File

@@ -0,0 +1,167 @@
/*
* Copyright 2010-2026 Branimir Karadzic. All rights reserved.
* License: https://github.com/bkaradzic/bx/blob/master/LICENSE
*/
#include <bx/scanner.h>
namespace bx
{
bool Scanner::seek(const StringView& _to)
{
if (contain(m_input, _to) )
{
moveTo(_to);
return true;
}
BX_TRACE("StringView '%S' is not a view into input string!", &_to);
return false;
}
uint32_t Scanner::getColumn() const
{
return uint32_t(between(getCursor(Cursor::LineStart) ).getLength() ) + 1;
}
StringView Scanner::getCursor(Cursor _which) const
{
const char* ptr = m_tail.getPtr();
if (Cursor::LineStart == _which)
{
while (ptr > m_input.getPtr()
&& '\n' != ptr[-1])
{
--ptr;
}
}
else if (Cursor::LineEnd == _which)
{
const char* term = m_input.getTerm();
while (ptr < term
&& '\n' != *ptr)
{
++ptr;
}
}
return StringView(ptr, ptr);
}
StringView Scanner::moveTo(const StringView& _to)
{
if (BX_LIKELY(!_to.isEmpty() ) )
{
if (BX_LIKELY(overlap(m_tail, _to) ) )
{
const StringView result(m_tail.getPtr(), _to.getTerm() );
m_tail.set(_to.getTerm(), m_tail.getTerm() );
m_line += countLines(result);
return result;
}
const StringView result(_to.getPtr(), m_tail.getPtr() );
m_tail.set(_to.getPtr(), m_tail.getTerm() );
m_line -= countLines(result);
return result;
}
return StringView();
}
StringView Scanner::strFunc(Class _class) const
{
switch (_class)
{
case Class::Space:
return strLTrimSpace(m_tail);
case Class::NonSpace:
return strLTrimNonSpace(m_tail);
case Class::Identifier:
{
const StringView word = strWord(m_tail);
return word.isEmpty()
? m_tail
: StringView(word.getTerm(), m_tail.getTerm() )
;
}
case Class::EndOfLine:
return strFindEol(m_tail);
case Class::NewLine:
return strFindNl(m_tail);
default:
BX_ASSERT(false, "Bug, _class can't be %d!", _class);
BX_UNREACHABLE;
}
}
StringView Scanner::moveBy(int32_t _bytes)
{
const int32_t len = m_tail.getLength();
_bytes = clamp(_bytes, len - m_input.getLength(), len);
if (BX_LIKELY(0 < _bytes) )
{
const StringView result(m_tail.getPtr(), m_tail.getPtr() + _bytes);
m_tail.set(result.getTerm(), m_tail.getTerm() );
m_line += countLines(result);
return result;
}
const StringView result(m_tail.getPtr() + _bytes, m_tail.getPtr() );
m_tail.set(result.getPtr(), m_tail.getTerm() );
m_line -= countLines(result);
return result;
}
uint32_t Scanner::countLines(const StringView& _str)
{
uint32_t line = 0;
StringView str = strFindEol(_str);
while (!str.isEmpty() )
{
++line;
str = strFindEol(strFindNl(str) );
}
return line;
}
StringView LineReader::next()
{
if (!m_scanner.isDone() )
{
++m_line;
const StringView cursor = m_scanner.getCursor();
m_scanner.acceptUntil(Scanner::Class::EndOfLine);
const StringView line = m_scanner.between(cursor);
m_scanner.accept(Scanner::Class::NewLine);
// A line terminator is `\n` or `\r\n`, and `Class::EndOfLine` already
// excludes it. Only malformed input (`\r` at the end of input, or
// `\r\r\n`) can leave a trailing `\r` behind.
return strRTrim(line, "\r");
}
return m_scanner.getCursor();
}
} // namespace bx

655
src/settings.cpp
View File

@@ -4,209 +4,496 @@
*/
#include <bx/settings.h>
namespace
{
#define INI_MALLOC(_ctx, _size) (bx::alloc(reinterpret_cast<bx::AllocatorI*>(_ctx), _size) )
#define INI_FREE(_ctx, _ptr) (bx::free(reinterpret_cast<bx::AllocatorI*>(_ctx), _ptr) )
#define INI_MEMCPY(_dst, _src, _count) (bx::memCopy(_dst, _src, _count) )
#define INI_STRLEN(_str) (bx::strLen(_str) )
#define INI_STRNICMP(_s1, _s2, _len) (bx::strCmpI(_s1, _s2, _len) )
#define INI_IMPLEMENTATION
BX_PRAGMA_DIAGNOSTIC_IGNORED_CLANG_GCC("-Wunused-function");
BX_PRAGMA_DIAGNOSTIC_PUSH();
BX_PRAGMA_DIAGNOSTIC_IGNORED_CLANG_GCC("-Wsign-compare");
#include <ini/ini.h>
BX_PRAGMA_DIAGNOSTIC_POP();
}
#include <bx/filepath.h>
#include <bx/scanner.h>
namespace bx
{
Settings::Settings(AllocatorI* _allocator, const void* _data, uint32_t _len)
: m_allocator(_allocator)
, m_ini(NULL)
{
load(_data, _len);
}
#define INI_T(_ptr) reinterpret_cast<ini_t*>(_ptr)
Settings::~Settings()
{
ini_destroy(INI_T(m_ini) );
}
void Settings::clear()
{
load(NULL, 0);
}
void Settings::load(const void* _data, uint32_t _len)
{
if (NULL != m_ini)
static StringView settingsAlloc(AllocatorI* _allocator, const StringView& _str)
{
ini_destroy(INI_T(m_ini) );
}
const int32_t len = _str.getLength();
if (NULL == _data)
{
m_ini = ini_create(m_allocator);
}
else
{
m_ini = ini_load( (const char*)_data, _len, m_allocator);
}
}
StringView Settings::get(const StringView& _name) const
{
ini_t* ini = INI_T(m_ini);
FilePath uri(_name);
const StringView path(strTrim(uri.getPath(), "/") );
const StringView& fileName(uri.getFileName() );
int32_t section = INI_GLOBAL_SECTION;
if (!path.isEmpty() )
{
section = ini_find_section(ini, path.getPtr(), path.getLength() );
if (INI_NOT_FOUND == section)
if (0 == len)
{
section = INI_GLOBAL_SECTION;
return StringView();
}
char* ptr = (char*)alloc(_allocator, len);
memCopy(ptr, _str.getPtr(), len);
return StringView(ptr, len);
}
static void settingsFree(AllocatorI* _allocator, StringView& _str)
{
if (!_str.isEmpty() )
{
free(_allocator, const_cast<char*>(_str.getPtr() ) );
}
_str.clear();
}
static void settingsAssign(AllocatorI* _allocator, StringView& _dst, const StringView& _src)
{
const StringView next = settingsAlloc(_allocator, _src);
settingsFree(_allocator, _dst);
_dst = next;
}
struct Settings::Ini
{
static constexpr int32_t kGlobal = 0;
static constexpr int32_t kInvalid = -1;
struct Property
{
StringView name;
StringView value;
};
struct Section
{
StringView name;
Property* props;
uint32_t count;
uint32_t capacity;
};
void init(AllocatorI* _allocator)
{
m_allocator = _allocator;
m_sections = NULL;
m_count = 0;
m_capacity = 0;
clear();
}
void shutdown()
{
reset();
if (NULL != m_sections)
{
free(m_allocator, m_sections);
m_sections = NULL;
}
m_capacity = 0;
}
void clear()
{
reset();
addSection(StringView() );
}
void load(const StringView& _data)
{
clear();
Scanner scanner(_data);
int32_t section = kGlobal;
while (!scanner.isDone() )
{
scanner.accept(Scanner::Class::Space);
if (scanner.isDone() )
{
break;
}
Scanner line(scanner.acceptUntil(Scanner::Class::EndOfLine) );
// Line comment.
if (!line.accept(';').isEmpty() )
{
continue;
}
// [section] header.
if (!line.accept('[').isEmpty() )
{
line.accept(Scanner::Class::Space);
const StringView name = strRTrimSpace(line.acceptUntil("]") );
if (!line.accept(']').isEmpty()
&& !name.isEmpty() )
{
const int32_t existing = findSection(name);
section = kInvalid == existing ? addSection(name) : existing;
}
continue;
}
// name = value property.
const StringView name = strRTrimSpace(line.acceptUntil("=") );
if (line.accept('=').isEmpty()
|| name.isEmpty() )
{
continue;
}
line.accept(Scanner::Class::Space);
setProperty(section, name, strRTrimSpace(line.acceptAll() ) );
}
}
int32_t find(const StringView& _name, StringView& _propertyName) const
{
const FilePath uri(_name);
const StringView path(strTrim(uri.getPath(), "/") );
_propertyName = uri.getFileName();
if (path.isEmpty() )
{
return kGlobal;
}
const int32_t section = findSection(path);
return kInvalid == section ? kGlobal : section;
}
int32_t findSection(const StringView& _name) const
{
for (uint32_t ss = 0; ss < m_count; ++ss)
{
const StringView& name = m_sections[ss].name;
if (isEqual(name, _name, false) )
{
return int32_t(ss);
}
}
return kInvalid;
}
int32_t addSection(const StringView& _name)
{
if (m_count == m_capacity)
{
m_capacity = 0 == m_capacity ? 8 : m_capacity * 2;
m_sections = (Section*)realloc(m_allocator, m_sections, m_capacity * sizeof(Section) );
}
Section& section = m_sections[m_count];
section.name = settingsAlloc(m_allocator, _name);
section.props = NULL;
section.count = 0;
section.capacity = 0;
return int32_t(m_count++);
}
void removeSection(int32_t _section)
{
if (kGlobal >= _section
|| uint32_t(_section) >= m_count)
{
return;
}
freeSection(m_sections[_section]);
for (uint32_t ss = uint32_t(_section); ss+1 < m_count; ++ss)
{
m_sections[ss] = m_sections[ss+1];
}
--m_count;
}
int32_t findProperty(int32_t _section, const StringView& _name) const
{
if (0 > _section
|| uint32_t(_section) >= m_count)
{
return kInvalid;
}
const Section& section = m_sections[_section];
for (uint32_t pp = 0; pp < section.count; ++pp)
{
const StringView& name = section.props[pp].name;
if (isEqual(name, _name, false) )
{
return int32_t(pp);
}
}
return kInvalid;
}
void setProperty(int32_t _section, const StringView& _name, const StringView& _value)
{
if (0 > _section
|| uint32_t(_section) >= m_count)
{
return;
}
const int32_t property = findProperty(_section, _name);
Section& section = m_sections[_section];
if (kInvalid != property)
{
settingsAssign(m_allocator, section.props[property].value, _value);
return;
}
if (section.count == section.capacity)
{
section.capacity = 0 == section.capacity ? 8 : section.capacity * 2;
section.props = (Property*)realloc(m_allocator, section.props, section.capacity * sizeof(Property) );
}
Property& prop = section.props[section.count];
prop.name = settingsAlloc(m_allocator, _name);
prop.value = settingsAlloc(m_allocator, _value);
++section.count;
}
void removeProperty(int32_t _section, int32_t _property)
{
if (0 > _section
|| uint32_t(_section) >= m_count)
{
return;
}
Section& section = m_sections[_section];
if (0 > _property
|| uint32_t(_property) >= section.count)
{
return;
}
settingsFree(m_allocator, section.props[_property].name);
settingsFree(m_allocator, section.props[_property].value);
for (uint32_t pp = uint32_t(_property); pp+1 < section.count; ++pp)
{
section.props[pp] = section.props[pp+1];
}
--section.count;
}
StringView getPropertyValue(int32_t _section, int32_t _property) const
{
if (0 > _section
|| uint32_t(_section) >= m_count)
{
return StringView();
}
const Section& section = m_sections[_section];
return (0 <= _property && uint32_t(_property) < section.count)
? section.props[_property].value
: StringView()
;
}
uint32_t getPropertyCount(int32_t _section) const
{
return (0 <= _section && uint32_t(_section) < m_count)
? m_sections[_section].count
: 0
;
}
int32_t save(WriterI* _writer, Error* _err) const
{
int32_t total = 0;
for (uint32_t ss = 0; ss < m_count; ++ss)
{
const Section& section = m_sections[ss];
if (!section.name.isEmpty() )
{
total += bx::write(_writer, "[", _err);
total += bx::write(_writer, section.name, _err);
total += bx::write(_writer, "]\n", _err);
}
for (uint32_t pp = 0; pp < section.count; ++pp)
{
total += bx::write(_writer, section.props[pp].name, _err);
total += bx::write(_writer, "=", _err);
total += bx::write(_writer, section.props[pp].value, _err);
total += bx::write(_writer, "\n", _err);
}
if (0 != total)
{
total += bx::write(_writer, "\n", _err);
}
}
return total;
}
void freeSection(Section& _section)
{
for (uint32_t pp = 0; pp < _section.count; ++pp)
{
settingsFree(m_allocator, _section.props[pp].name);
settingsFree(m_allocator, _section.props[pp].value);
}
if (NULL != _section.props)
{
free(m_allocator, _section.props);
}
settingsFree(m_allocator, _section.name);
_section.props = NULL;
_section.count = 0;
_section.capacity = 0;
}
void reset()
{
for (uint32_t ss = 0; ss < m_count; ++ss)
{
freeSection(m_sections[ss]);
}
m_count = 0;
}
AllocatorI* m_allocator;
Section* m_sections;
uint32_t m_count;
uint32_t m_capacity;
};
Settings::Settings(AllocatorI* _allocator, const void* _data, uint32_t _len)
: m_allocator(_allocator)
, m_ini(BX_NEW(_allocator, Ini) )
{
m_ini->init(_allocator);
load(_data, _len);
}
Settings::~Settings()
{
m_ini->shutdown();
deleteObject(m_allocator, m_ini);
}
void Settings::clear()
{
m_ini->clear();
}
void Settings::load(const void* _data, uint32_t _len)
{
if (NULL == _data
|| 0 == _len)
{
m_ini->clear();
return;
}
m_ini->load(StringView( (const char*)_data, int32_t(_len) ) );
}
StringView Settings::get(const StringView& _name) const
{
StringView propertyName;
const int32_t section = m_ini->find(_name, propertyName);
return m_ini->getPropertyValue(section, m_ini->findProperty(section, propertyName) );
}
void Settings::set(const StringView& _name, const StringView& _value)
{
const FilePath uri(_name);
const StringView path(strTrim(uri.getPath(), "/") );
const StringView& fileName(uri.getFileName() );
int32_t section = Ini::kGlobal;
if (!path.isEmpty() )
{
section = m_ini->findSection(path);
if (Ini::kInvalid == section)
{
section = m_ini->addSection(path);
}
}
m_ini->setProperty(section, fileName, _value);
}
void Settings::remove(const StringView& _name) const
{
StringView propertyName;
const int32_t section = m_ini->find(_name, propertyName);
const int32_t property = m_ini->findProperty(section, propertyName);
if (Ini::kInvalid == property)
{
return;
}
m_ini->removeProperty(section, property);
if (Ini::kGlobal != section
&& 0 == m_ini->getPropertyCount(section) )
{
m_ini->removeSection(section);
}
}
int32_t property = ini_find_property(ini, section, fileName.getPtr(), fileName.getLength() );
if (INI_NOT_FOUND == property)
int32_t Settings::read(ReaderSeekerI* _reader, Error* _err)
{
return StringView();
int32_t size = int32_t(getRemain(_reader) );
void* data = bx::alloc(m_allocator, size);
int32_t total = bx::read(_reader, data, size, _err);
load(data, size);
bx::free(m_allocator, data);
return total;
}
return ini_property_value(ini, section, property);
}
void Settings::set(const StringView& _name, const StringView& _value)
{
ini_t* ini = INI_T(m_ini);
FilePath uri(_name);
const StringView path(strTrim(uri.getPath(), "/") );
const StringView& fileName(uri.getFileName() );
int32_t section = INI_GLOBAL_SECTION;
if (!path.isEmpty() )
int32_t Settings::write(WriterI* _writer, Error* _err) const
{
section = ini_find_section(ini, path.getPtr(), path.getLength() );
if (INI_NOT_FOUND == section)
{
section = ini_section_add(ini, path.getPtr(), path.getLength() );
}
return m_ini->save(_writer, _err);
}
int32_t property = ini_find_property(ini, section, fileName.getPtr(), fileName.getLength() );
if (INI_NOT_FOUND == property)
int32_t read(ReaderSeekerI* _reader, Settings& _settings, Error* _err)
{
ini_property_add(
ini
, section
, fileName.getPtr()
, fileName.getLength()
, _value.getPtr()
, _value.getLength()
);
}
else
{
ini_property_value_set(
ini
, section
, property
, _value.getPtr()
, _value.getLength()
);
}
}
void Settings::remove(const StringView& _name) const
{
ini_t* ini = INI_T(m_ini);
FilePath uri(_name);
const StringView path = strTrim(uri.getPath(), "/");
const StringView& fileName = uri.getFileName();
int32_t section = INI_GLOBAL_SECTION;
if (!path.isEmpty() )
{
section = ini_find_section(ini, path.getPtr(), path.getLength() );
if (INI_NOT_FOUND == section)
{
section = INI_GLOBAL_SECTION;
}
BX_ERROR_SCOPE(_err);
return _settings.read(_reader, _err);
}
int32_t property = ini_find_property(ini, section, fileName.getPtr(), fileName.getLength() );
if (INI_NOT_FOUND == property)
int32_t write(WriterI* _writer, const Settings& _settings, Error* _err)
{
return;
BX_ERROR_SCOPE(_err);
return _settings.write(_writer, _err);
}
ini_property_remove(ini, section, property);
if (INI_GLOBAL_SECTION != section
&& 0 == ini_property_count(ini, section) )
{
ini_section_remove(ini, section);
}
}
int32_t Settings::read(ReaderSeekerI* _reader, Error* _err)
{
int32_t size = int32_t(getRemain(_reader) );
void* data = bx::alloc(m_allocator, size);
int32_t total = bx::read(_reader, data, size, _err);
load(data, size);
bx::free(m_allocator, data);
return total;
}
int32_t Settings::write(WriterI* _writer, Error* _err) const
{
ini_t* ini = INI_T(m_ini);
int32_t size = ini_save(ini, NULL, 0);
void* data = bx::alloc(m_allocator, size);
ini_save(ini, (char*)data, size);
int32_t total = bx::write(_writer, data, size-1, _err);
bx::free(m_allocator, data);
return total;
}
#undef INI_T
int32_t read(ReaderSeekerI* _reader, Settings& _settings, Error* _err)
{
BX_ERROR_SCOPE(_err);
return _settings.read(_reader, _err);
}
int32_t write(WriterI* _writer, const Settings& _settings, Error* _err)
{
BX_ERROR_SCOPE(_err);
return _settings.write(_writer, _err);
}
} // namespace bx

93
src/string.cpp
View File

@@ -10,65 +10,6 @@
namespace bx
{
inline bool isInRange(char _ch, char _from, char _to)
{
return unsigned(_ch - _from) <= unsigned(_to-_from);
}
bool isSpace(char _ch)
{
return ' ' == _ch // Space.
|| '\t' == _ch // Horizontal tab.
|| '\n' == _ch // Line feed / new line.
|| '\r' == _ch // Carriage return.
|| '\v' == _ch // Vertical tab.
|| '\f' == _ch // Form feed / new page.
;
}
bool isUpper(char _ch)
{
return isInRange(_ch, 'A', 'Z');
}
bool isLower(char _ch)
{
return isInRange(_ch, 'a', 'z');
}
bool isAlpha(char _ch)
{
return isLower(_ch) || isUpper(_ch);
}
bool isNumeric(char _ch)
{
return isInRange(_ch, '0', '9');
}
bool isAlphaNum(char _ch)
{
return false
|| isAlpha(_ch)
|| isNumeric(_ch)
;
}
bool isHexNum(char _ch)
{
return false
|| isInRange(toLower(_ch), 'a', 'f')
|| isNumeric(_ch)
;
}
bool isPrint(char _ch)
{
return isInRange(_ch, ' ', '~');
}
typedef bool (*CharTestFn)(char _ch);
template<CharTestFn fn>
inline bool isCharTest(const StringView& _str)
{
@@ -125,11 +66,6 @@ namespace bx
return isCharTest<isPrint>(_str);
}
char toLower(char _ch)
{
return _ch + (isUpper(_ch) ? 0x20 : 0);
}
void toLowerUnsafe(char* _inOutStr, int32_t _len)
{
for (int32_t ii = 0; ii < _len; ++ii)
@@ -144,11 +80,6 @@ namespace bx
toLowerUnsafe(_inOutStr, len);
}
char toUpper(char _ch)
{
return _ch - (isLower(_ch) ? 0x20 : 0);
}
void toUpperUnsafe(char* _inOutStr, int32_t _len)
{
for (int32_t ii = 0; ii < _len; ++ii)
@@ -165,7 +96,7 @@ namespace bx
typedef char (*CharFn)(char _ch);
inline constexpr char toNoop(char _ch)
inline BX_CONSTEXPR_FUNC char toNoop(char _ch)
{
return _ch;
}
@@ -560,8 +491,6 @@ namespace bx
);
}
constexpr uint32_t kFindStep = 1024;
StringView strFindNl(const StringView& _str)
{
StringView str(_str);
@@ -578,23 +507,19 @@ namespace bx
StringView strFindEol(const StringView& _str)
{
StringView str(_str);
const StringView eol = strFind(_str, '\n');
for (; str.getPtr() != _str.getTerm()
; str = StringView(min(str.getPtr() + kFindStep, _str.getTerm() ), min(str.getPtr() + kFindStep*2, _str.getTerm() ) )
)
if (!eol.isEmpty() )
{
StringView eol = strFind(str, "\r\n");
if (!eol.isEmpty() )
const char* ptr = eol.getPtr();
if (ptr != _str.getPtr()
&& '\r' == ptr[-1])
{
return StringView(eol.getPtr(), _str.getTerm() );
--ptr;
}
eol = strFind(str, '\n');
if (!eol.isEmpty() )
{
return StringView(eol.getPtr(), _str.getTerm() );
}
return StringView(ptr, _str.getTerm() );
}
return StringView(_str.getTerm(), _str.getTerm() );

149
src/url.cpp
View File

@@ -3,10 +3,36 @@
* License: https://github.com/bkaradzic/bnet#license-bsd-2-clause
*/
#include <bx/scanner.h>
#include <bx/url.h>
namespace bx
{
static bool isNotSlash(char _ch)
{
return '/' != _ch;
}
static bool isNotColon(char _ch)
{
return ':' != _ch;
}
static bool isNotQuery(char _ch)
{
return '?' != _ch;
}
static bool isNotFragment(char _ch)
{
return '#' != _ch;
}
static bool isNotQueryOrFragment(char _ch)
{
return isNotQuery(_ch) && isNotFragment(_ch);
}
UrlView::UrlView()
{
}
@@ -23,22 +49,16 @@ namespace bx
{
clear();
const char* term = _url.getTerm();
StringView schemeEnd = strFind(_url, "://");
const char* hostStart = !schemeEnd.isEmpty() ? schemeEnd.getTerm() : _url.getPtr();
StringView path = strFind(StringView(hostStart, term), '/');
Scanner scanner(_url);
if (schemeEnd.isEmpty()
&& path.isEmpty() )
const StringView schemeBegin = scanner.getCursor();
scanner.acceptUntil("://");
const StringView scheme = scanner.between(schemeBegin);
const bool hasScheme = !scanner.accept("://").isEmpty();
if (hasScheme)
{
return false;
}
if (!schemeEnd.isEmpty()
&& (path.isEmpty() || path.getPtr() > schemeEnd.getPtr() ) )
{
const StringView scheme(_url.getPtr(), schemeEnd.getPtr() );
if (!isAlpha(scheme) )
{
return false;
@@ -47,65 +67,72 @@ namespace bx
m_tokens[Scheme].set(scheme);
}
if (!path.isEmpty() )
{
path.set(path.getPtr(), term);
const StringView query = strFind(path, '?');
const StringView fragment = strFind(path, '#');
const StringView authorityBegin = scanner.getCursor();
scanner.acceptWhile(isNotSlash);
const StringView authority = scanner.between(authorityBegin);
if (!fragment.isEmpty()
&& fragment.getPtr() < query.getPtr() )
const bool hasPath = !scanner.peek('/').isEmpty();
if (!hasScheme
&& !hasPath)
{
return false;
}
if (hasPath)
{
const StringView pathBegin = scanner.getCursor();
scanner.acceptWhile(isNotQueryOrFragment);
m_tokens[Path].set(scanner.between(pathBegin) );
if (!scanner.accept('?').isEmpty() )
{
const StringView queryBegin = scanner.getCursor();
scanner.acceptWhile(isNotFragment);
m_tokens[Query].set(scanner.between(queryBegin) );
}
if (!scanner.accept('#').isEmpty() )
{
const StringView fragmentBegin = scanner.getCursor();
scanner.acceptWhile(isNotQuery);
m_tokens[Fragment].set(scanner.between(fragmentBegin) );
}
// Anything left over is a query following a fragment.
if (!scanner.isDone() )
{
return false;
}
m_tokens[Path].set(path.getPtr()
, !query.isEmpty() ? query.getPtr()
: !fragment.isEmpty() ? fragment.getPtr()
: term
);
if (!query.isEmpty() )
{
m_tokens[Query].set(query.getPtr()+1
, !fragment.isEmpty() ? fragment.getPtr()
: term
);
}
if (!fragment.isEmpty() )
{
m_tokens[Fragment].set(fragment.getPtr()+1, term);
}
term = path.getPtr();
}
const StringView userPassEnd = strFind(StringView(hostStart, term), '@');
const char* userPassStart = !userPassEnd.isEmpty() ? hostStart : NULL;
hostStart = !userPassEnd.isEmpty() ? userPassEnd.getPtr()+1 : hostStart;
const StringView portStart = strFind(StringView(hostStart, term), ':');
Scanner authorityScanner(authority);
m_tokens[Host].set(hostStart, !portStart.isEmpty() ? portStart.getPtr() : term);
const StringView userInfoBegin = authorityScanner.getCursor();
authorityScanner.acceptUntil("@");
const StringView userInfo = authorityScanner.between(userInfoBegin);
if (!portStart.isEmpty())
if (!authorityScanner.accept('@').isEmpty() )
{
m_tokens[Port].set(portStart.getPtr()+1, term);
Scanner userInfoScanner(userInfo);
const StringView userNameBegin = userInfoScanner.getCursor();
userInfoScanner.acceptWhile(isNotColon);
m_tokens[UserName].set(userInfoScanner.between(userNameBegin) );
if (!userInfoScanner.accept(':').isEmpty() )
{
m_tokens[Password].set(userInfoScanner.acceptAll() );
}
}
if (NULL != userPassStart)
const StringView hostBegin = authorityScanner.getCursor();
authorityScanner.acceptWhile(isNotColon);
m_tokens[Host].set(authorityScanner.between(hostBegin) );
if (!authorityScanner.accept(':').isEmpty() )
{
StringView passStart = strFind(StringView(userPassStart, userPassEnd.getPtr() ), ':');
m_tokens[UserName].set(userPassStart
, !passStart.isEmpty() ? passStart.getPtr()
: userPassEnd.getPtr()
);
if (!passStart.isEmpty() )
{
m_tokens[Password].set(passStart.getPtr()+1, userPassEnd.getPtr() );
}
m_tokens[Port].set(authorityScanner.acceptAll() );
}
return true;

28
tests/filepath_test.cpp
View File

@@ -64,6 +64,15 @@ FilePathTest s_filePathTest[] =
{"abc\\/../..\\/././../def", "../../def"},
{"\\abc/def\\../..\\..", "/"},
// Drive letter
{"c:", "C:"},
{"c:/", "C:/"},
{"c:abc", "C:/abc"},
{"c:/abc/../def", "C:/def"},
{"c:/..", "C:/"},
{"c:/../..", "C:/"},
{"c:\\abc\\..\\def", "C:/def"},
};
struct FilePathSplit
@@ -118,6 +127,25 @@ TEST_CASE("FilePath", "[filepath][string]")
};
}
TEST_CASE("FilePath sub-view", "[filepath]")
{
const char buffer[] = "C:/abc";
bx::FilePath fp;
fp.set(bx::StringView(buffer, 2) );
REQUIRE(0 == bx::strCmp("C:", fp) );
const char buffer2[] = "ab/cd";
fp.set(bx::StringView(buffer2, 2) );
REQUIRE(0 == bx::strCmp("ab", fp) );
fp.set(bx::StringView(buffer2, 3) );
REQUIRE(0 == bx::strCmp("ab/", fp) );
fp.set(bx::StringView(buffer2, 0) );
REQUIRE(0 == bx::strCmp(".", fp) );
}
TEST_CASE("FilePath temp", "[filepath]")
{
bx::FilePath tmp(bx::Dir::Temp);

351
tests/scanner_test.cpp Normal file
View File

@@ -0,0 +1,351 @@
/*
* Copyright 2010-2026 Branimir Karadzic. All rights reserved.
* License: https://github.com/bkaradzic/bx#license-bsd-2-clause
*/
#include "test.h"
#include <bx/scanner.h>
namespace bx
{
uint32_t countLines(const StringView& _str)
{
Scanner scanner(_str);
scanner.acceptAll();
return scanner.getLine();
}
void printLines(const StringView& _str)
{
Scanner scanner(_str);
while (!scanner.isDone() )
{
StringView sv = scanner.acceptUntil(Scanner::Class::EndOfLine);
BX_TRACE("%d: '%S'", scanner.getLine(), &sv);
scanner.acceptUntil(Scanner::Class::NewLine);
}
}
} // namespace bx
TEST_CASE("Scanner", "[scanner]")
{
const bx::StringLiteral input =
"test (foo) bar [baz]xyz\n"
" // test\n"
"tset (foo) bar [baz]xyz\n"
" \n"
"void main()\n"
"{\n"
"\treturn;\n"
"}\n"
;
const uint32_t numLines = bx::countLines(input);
bx::printLines(input);
bx::Scanner scanner(input);
REQUIRE(!scanner.isDone() );
bx::StringView start = scanner.accept();
REQUIRE("t" == start);
REQUIRE("est " == scanner.acceptUntil("(") );
REQUIRE("(" == scanner.accept('(') );
REQUIRE(!scanner.isDone() );
REQUIRE("foo" == scanner.accept(bx::Scanner::Class::Identifier) );
REQUIRE(")" == scanner.accept(')') );
REQUIRE(!scanner.isDone() );
REQUIRE(" " == scanner.accept(' ') );
REQUIRE(!scanner.isDone() );
REQUIRE("b" == scanner.accept('r', 'b', 'a') );
REQUIRE("a" == scanner.accept('b', 'a', 'r') );
REQUIRE("r" == scanner.accept('a', 'r', 'b') );
REQUIRE(!scanner.isDone() );
REQUIRE(" " == scanner.accept(' ') );
REQUIRE(!scanner.isDone() );
REQUIRE("[baz]" == scanner.acceptWhile("abz[]") );
REQUIRE(!scanner.isDone() );
REQUIRE("xyz" == scanner.accept("xyz") );
REQUIRE(0 == scanner.getLine() );
REQUIRE("\n " == scanner.acceptUntil(bx::Scanner::Class::Space) );
REQUIRE(1 == scanner.getLine() );
scanner.acceptAll();
REQUIRE(scanner.accept().isEmpty() );
REQUIRE(numLines == scanner.getLine() );
REQUIRE(scanner.isDone() );
REQUIRE(scanner.seek(start) );
REQUIRE(0 == scanner.getLine() );
REQUIRE(!scanner.isDone() );
REQUIRE(scanner.seek(INT32_MAX) );
REQUIRE(numLines == scanner.getLine() );
REQUIRE(scanner.isDone() );
REQUIRE(scanner.seek(INT32_MIN) );
REQUIRE(0 == scanner.getLine() );
REQUIRE(!scanner.isDone() );
}
TEST_CASE("Scanner.class accept", "[scanner]")
{
{
bx::Scanner sc(bx::StringView("abc def") );
REQUIRE("abc" == sc.accept(bx::Scanner::Class::Identifier) );
REQUIRE(" def" == sc.acceptAll() );
}
{
bx::Scanner sc(bx::StringView(" abc def") );
REQUIRE(" " == sc.accept(bx::Scanner::Class::Space) );
REQUIRE(!sc.isDone() );
REQUIRE("abc def" == sc.acceptAll() );
}
{
bx::Scanner sc(bx::StringView("abc") );
REQUIRE(sc.accept(bx::Scanner::Class::Space).isEmpty() );
REQUIRE("abc" == sc.acceptAll() );
}
{
bx::Scanner sc(bx::StringView("abc def") );
REQUIRE("abc" == sc.accept(bx::Scanner::Class::NonSpace) );
REQUIRE(" def" == sc.acceptAll() );
}
{
bx::Scanner sc(bx::StringView("abc\ndef") );
REQUIRE("abc" == sc.accept(bx::Scanner::Class::EndOfLine) );
REQUIRE("\ndef" == sc.acceptAll() );
}
{
bx::Scanner a(bx::StringView(" abc") );
bx::Scanner b(bx::StringView(" abc") );
REQUIRE(" " == a.accept(bx::Scanner::Class::Space) );
REQUIRE(" " == b.acceptUntil(bx::Scanner::Class::Space) );
}
}
TEST_CASE("Scanner.peek", "[scanner]")
{
bx::Scanner sc(bx::StringView("abc def") );
REQUIRE("a" == sc.peek() );
REQUIRE("a" == sc.peek('a') );
REQUIRE(sc.peek('x').isEmpty() );
REQUIRE("abc" == sc.peek(bx::StringView("abc") ) );
REQUIRE(sc.peek(bx::StringView("abx") ).isEmpty() );
REQUIRE("abc" == sc.peek(bx::Scanner::Class::Identifier) );
REQUIRE("a" == sc.accept() );
REQUIRE("bc" == sc.peek(bx::Scanner::Class::Identifier) );
REQUIRE("bc" == sc.accept(bx::Scanner::Class::Identifier) );
sc.acceptAll();
REQUIRE(sc.isDone() );
REQUIRE(sc.peek().isEmpty() );
REQUIRE(sc.peek(bx::Scanner::Class::Identifier).isEmpty() );
}
TEST_CASE("Scanner.getCursor between", "[scanner]")
{
bx::Scanner sc(bx::StringView("abc def") );
const bx::StringView mark = sc.getCursor();
REQUIRE(mark.isEmpty() );
sc.accept(bx::Scanner::Class::Identifier);
REQUIRE("abc" == sc.between(mark) );
const bx::StringView here = sc.getCursor();
sc.acceptAll();
REQUIRE(" def" == sc.between(here) );
REQUIRE(sc.seek(INT32_MIN) );
REQUIRE("abc" == sc.between(here) );
}
TEST_CASE("Scanner.getCursor lines", "[scanner]")
{
bx::Scanner sc(bx::StringView("abc\ndefgh\nij") );
sc.accept(bx::Scanner::Class::Identifier);
sc.accept('\n');
sc.accept('d');
sc.accept('e');
REQUIRE("de" == sc.between(sc.getCursor(bx::Scanner::Cursor::LineStart) ) );
REQUIRE("fgh" == sc.between(sc.getCursor(bx::Scanner::Cursor::LineEnd) ) );
}
TEST_CASE("Scanner.getColumn", "[scanner]")
{
bx::Scanner sc(bx::StringView("abc\ndef") );
REQUIRE(1 == sc.getColumn() );
sc.accept();
REQUIRE(2 == sc.getColumn() );
sc.accept(bx::Scanner::Class::Identifier);
REQUIRE(4 == sc.getColumn() );
sc.accept('\n');
REQUIRE(1 == sc.getLine() );
REQUIRE(1 == sc.getColumn() );
sc.accept();
REQUIRE(2 == sc.getColumn() );
}
TEST_CASE("Scanner.predicate accept", "[scanner]")
{
bx::Scanner sc(bx::StringView("123abc xyz") );
REQUIRE("1" == sc.accept(bx::isNumeric) );
REQUIRE(sc.accept(bx::isAlpha).isEmpty() );
REQUIRE("2" == sc.accept(bx::isNumeric) );
REQUIRE("3" == sc.acceptWhile(bx::isNumeric) );
REQUIRE("abc" == sc.acceptWhile(bx::isAlpha) );
REQUIRE(sc.acceptWhile(bx::isNumeric).isEmpty() );
REQUIRE(!sc.isDone() );
REQUIRE(" " == sc.acceptWhile(bx::isSpace) );
REQUIRE("xyz" == sc.acceptWhile(bx::isAlphaNum) );
REQUIRE(sc.isDone() );
}
TEST_CASE("Scanner.reset", "[scanner]")
{
bx::Scanner sc(bx::StringView("abc\ndef\n") );
sc.acceptAll();
REQUIRE(sc.isDone() );
REQUIRE(2 == sc.getLine() );
sc.reset();
REQUIRE(!sc.isDone() );
REQUIRE(0 == sc.getLine() );
REQUIRE(1 == sc.getColumn() );
REQUIRE("abc" == sc.accept(bx::Scanner::Class::Identifier) );
// Reset from a partially consumed state.
sc.reset();
REQUIRE("abc\ndef\n" == sc.acceptAll() );
}
TEST_CASE("Scanner.EndOfLine mixed", "[scanner]")
{
// End of line must be the *first* terminator, even when a later line
// uses \r\n and an earlier one uses a lone \n.
bx::Scanner sc(bx::StringView("a\nb\r\n") );
REQUIRE("a" == sc.acceptUntil(bx::Scanner::Class::EndOfLine) );
REQUIRE("\n" == sc.accept(bx::Scanner::Class::NewLine) );
REQUIRE("b" == sc.acceptUntil(bx::Scanner::Class::EndOfLine) );
REQUIRE("\r\n" == sc.accept(bx::Scanner::Class::NewLine) );
REQUIRE(sc.isDone() );
REQUIRE(2 == bx::countLines(bx::StringView("a\nb\r\n") ) );
REQUIRE(4 == bx::countLines(bx::StringView("a\nb\r\nc\rd\ne\r\n") ) );
}
TEST_CASE("LineReader", "[scanner][string]")
{
{
bx::LineReader lr(bx::StringView("") );
REQUIRE(lr.isDone() );
REQUIRE(lr.next().isEmpty() );
REQUIRE(0 == lr.getLine() );
}
{
bx::LineReader lr(bx::StringView("abc") );
REQUIRE(!lr.isDone() );
REQUIRE("abc" == lr.next() );
REQUIRE(1 == lr.getLine() );
REQUIRE(lr.isDone() );
}
{
// \n, \r\n and empty lines interleaved.
bx::LineReader lr(bx::StringView("a\r\nb\n\nc\r\n\r\nd") );
REQUIRE("a" == lr.next() );
REQUIRE("b" == lr.next() );
REQUIRE(lr.next().isEmpty() );
REQUIRE("c" == lr.next() );
REQUIRE(lr.next().isEmpty() );
REQUIRE("d" == lr.next() );
REQUIRE(6 == lr.getLine() );
REQUIRE(lr.isDone() );
// next() past the end is a no-op.
REQUIRE(lr.next().isEmpty() );
REQUIRE(6 == lr.getLine() );
}
{
// Trailing \r without \n is trimmed, interior \r is not a line break.
bx::LineReader lr(bx::StringView("a\rb\nc\r") );
REQUIRE("a\rb" == lr.next() );
REQUIRE("c" == lr.next() );
REQUIRE(lr.isDone() );
}
{
// Only \r and \n are trimmed, other trailing whitespace is kept.
bx::LineReader lr(bx::StringView("a \r\n\tb\t\n") );
REQUIRE("a " == lr.next() );
REQUIRE("\tb\t" == lr.next() );
REQUIRE(lr.isDone() );
}
{
bx::LineReader lr(bx::StringView("a\nb\n") );
REQUIRE("a" == lr.next() );
REQUIRE("b" == lr.next() );
REQUIRE(lr.isDone() );
REQUIRE(2 == lr.getLine() );
lr.reset();
REQUIRE(!lr.isDone() );
REQUIRE(0 == lr.getLine() );
REQUIRE("a" == lr.next() );
REQUIRE("b" == lr.next() );
REQUIRE(2 == lr.getLine() );
}
{
// Returned views must point into the input, not at a temporary.
const bx::StringView input("abc\ndef\n");
bx::LineReader lr(input);
const bx::StringView first = lr.next();
REQUIRE(first.getPtr() == input.getPtr() );
const bx::StringView second = lr.next();
REQUIRE(second.getPtr() == input.getPtr() + 4);
}
}

177
tests/settings_test.cpp
View File

@@ -7,6 +7,40 @@
#include <bx/settings.h>
#include <bx/file.h>
class CountingAllocator : public bx::AllocatorI
{
public:
CountingAllocator()
: m_live(0)
{
}
virtual ~CountingAllocator()
{
}
virtual void* realloc(void* _ptr, size_t _size, size_t _align, const char* _filePath, uint32_t _line) override
{
if (NULL == _ptr
&& 0 != _size)
{
++m_live;
}
else if (NULL != _ptr
&& 0 == _size)
{
--m_live;
}
return m_allocator.realloc(_ptr, _size, _align, _filePath, _line);
}
int32_t m_live;
private:
bx::DefaultAllocator m_allocator;
};
TEST_CASE("Settings", "")
{
bx::FilePath filePath;
@@ -47,3 +81,146 @@ TEST_CASE("Settings", "")
REQUIRE(0 == bx::strCmp(settings.get("meh/podmac"), "true") );
REQUIRE(0 == bx::strCmp(settings.get("test/foo/bar/abvgd"), "1389") );
}
TEST_CASE("Settings parse", "")
{
bx::DefaultAllocator allocator;
const bx::StringView ini(
"; leading comment\n"
"globalKey=globalValue\n"
"\n"
"[section]\n"
"\tkey = value \n"
"spaces = a b c\n"
"; comment inside section\n"
"empty=\n"
"\n"
"[ padded ]\n"
"k=v\n"
"lineWithoutEquals\n"
"=valueWithoutName\n"
"\n"
"[dup]\n"
"x=1\n"
"x=2"
);
bx::Settings settings(&allocator, ini.getPtr(), ini.getLength() );
REQUIRE(0 == bx::strCmp(settings.get("globalKey"), "globalValue") );
REQUIRE(0 == bx::strCmp(settings.get("section/key"), "value") );
REQUIRE(0 == bx::strCmp(settings.get("section/spaces"), "a b c") );
REQUIRE(settings.get("section/empty").isEmpty() );
REQUIRE(0 == bx::strCmp(settings.get("padded/k"), "v") );
REQUIRE(settings.get("padded/lineWithoutEquals").isEmpty() );
REQUIRE(0 == bx::strCmp(settings.get("dup/x"), "2") );
REQUIRE(settings.get("section/; comment inside section").isEmpty() );
REQUIRE(0 == bx::strCmp(settings.get("SECTION/KEY"), "value") );
REQUIRE(settings.get("nosuch/key").isEmpty() );
REQUIRE(settings.get("nosuch").isEmpty() );
}
TEST_CASE("Settings parse CRLF", "")
{
bx::DefaultAllocator allocator;
const bx::StringView ini(
"[a]\r\n"
"key = value\r\n"
"\r\n"
"[b]\r\n"
"key=other\r\n"
);
bx::Settings settings(&allocator, ini.getPtr(), ini.getLength() );
REQUIRE(0 == bx::strCmp(settings.get("a/key"), "value") );
REQUIRE(0 == bx::strCmp(settings.get("b/key"), "other") );
}
TEST_CASE("Settings empty", "")
{
bx::DefaultAllocator allocator;
bx::Settings settings(&allocator);
REQUIRE(settings.get("anything").isEmpty() );
const bx::StringView comments("; just a comment\n\n\n \n");
settings.load(comments.getPtr(), comments.getLength() );
REQUIRE(settings.get("anything").isEmpty() );
settings.load(NULL, 0);
REQUIRE(settings.get("anything").isEmpty() );
}
TEST_CASE("Settings roundtrip", "")
{
bx::DefaultAllocator allocator;
bx::Settings settings(&allocator);
settings.set("global", "0");
settings.set("a/one", "1");
settings.set("a/two", "2");
settings.set("b/nested/three", "3");
char tmp[512];
bx::StaticMemoryBlockWriter writer(tmp, sizeof(tmp) );
const int32_t size = bx::write(&writer, settings, bx::ErrorIgnore{});
REQUIRE(0 < size);
bx::Settings reloaded(&allocator, tmp, uint32_t(size) );
REQUIRE(0 == bx::strCmp(reloaded.get("global"), "0") );
REQUIRE(0 == bx::strCmp(reloaded.get("a/one"), "1") );
REQUIRE(0 == bx::strCmp(reloaded.get("a/two"), "2") );
REQUIRE(0 == bx::strCmp(reloaded.get("b/nested/three"), "3") );
reloaded.set("a/one", "one-updated");
REQUIRE(0 == bx::strCmp(reloaded.get("a/one"), "one-updated") );
reloaded.remove("a/one");
reloaded.remove("a/two");
REQUIRE(reloaded.get("a/one").isEmpty() );
REQUIRE(reloaded.get("a/two").isEmpty() );
REQUIRE(0 == bx::strCmp(reloaded.get("b/nested/three"), "3") );
}
TEST_CASE("Settings allocations balanced", "")
{
CountingAllocator allocator;
{
const bx::StringView ini(
"global=0\n"
"[a]\n"
"one=1\n"
"two=2\n"
"[b]\n"
"three=3\n"
);
bx::Settings settings(&allocator, ini.getPtr(), ini.getLength() );
REQUIRE(0 < allocator.m_live);
settings.set("c/four", "4");
settings.set("a/one", "overwritten");
settings.remove("b/three");
settings.load(ini.getPtr(), ini.getLength() );
settings.clear();
settings.set("d/five", "5");
}
REQUIRE(0 == allocator.m_live);
}

49
tests/string_test.cpp
View File

@@ -691,6 +691,55 @@ TEST_CASE("strWord", "[string]")
REQUIRE(0 == bx::strCmp(bx::strWord("abvgd-1389.0"), "abvgd") );
}
TEST_CASE("strFindEol strFindNl", "[string]")
{
{
const bx::StringView test("abc");
REQUIRE(test.getTerm() == bx::strFindEol(test).getPtr() );
REQUIRE(test.getTerm() == bx::strFindNl(test).getPtr() );
}
{
const bx::StringView test("abc\ndef");
REQUIRE(test.getPtr() + 3 == bx::strFindEol(test).getPtr() );
REQUIRE(test.getPtr() + 4 == bx::strFindNl(test).getPtr() );
}
{
// End of line is the \r of a \r\n pair, new line is past the \n.
const bx::StringView test("abc\r\ndef");
REQUIRE(test.getPtr() + 3 == bx::strFindEol(test).getPtr() );
REQUIRE(test.getPtr() + 5 == bx::strFindNl(test).getPtr() );
}
{
// A lone \r is not a line terminator.
const bx::StringView test("abc\rdef\n");
REQUIRE(test.getPtr() + 7 == bx::strFindEol(test).getPtr() );
REQUIRE(test.getPtr() + 8 == bx::strFindNl(test).getPtr() );
}
{
// Must return the *first* terminator, not the first \r\n.
const bx::StringView test("abc\ndef\r\n");
REQUIRE(test.getPtr() + 3 == bx::strFindEol(test).getPtr() );
REQUIRE(test.getPtr() + 4 == bx::strFindNl(test).getPtr() );
}
{
const bx::StringView test("\r\n");
REQUIRE(test.getPtr() == bx::strFindEol(test).getPtr() );
REQUIRE(test.getTerm() == bx::strFindNl(test).getPtr() );
}
{
// Leading \n whose preceding \r is outside the view.
const bx::StringView test("a\r\nb");
const bx::StringView tail(test, 2, INT32_MAX);
REQUIRE(tail.getPtr() == bx::strFindEol(tail).getPtr() );
}
}
TEST_CASE("strFindBlock", "[string]")
{
const bx::StringView test0("{ { {} {} abvgd; {} } }");

33
tests/url_test.cpp
View File

@@ -41,6 +41,39 @@ static const UrlTest s_urlTest[] =
, "file:///d:/tmp/archive.tar.gz",
{ "file", "", "", "", "", "/d:/tmp/archive.tar.gz", "", "" },
},
{ true
, "scheme://host.rs/path#fragment",
{ "scheme", "", "", "host.rs", "", "/path", "", "fragment" },
},
{ true
, "scheme://host.rs/path?query#fragment",
{ "scheme", "", "", "host.rs", "", "/path", "query", "fragment" },
},
{ true
, "scheme://host.rs/path?",
{ "scheme", "", "", "host.rs", "", "/path", "", "" },
},
{ true
, "scheme://username:password@host.rs/",
{ "scheme", "username", "password", "host.rs", "", "/", "", "" },
},
{ true
, "scheme://host.rs",
{ "scheme", "", "", "host.rs", "", "", "", "" },
},
{ false // Fragment must not precede query.
, "scheme://host.rs/path#fragment?query",
{ "", "", "", "", "", "", "", "" },
},
{ false // Neither scheme nor path.
, "host.rs",
{ "", "", "", "", "", "", "", "" },
},
{ false // Scheme must be alpha.
, "1scheme://host.rs/",
{ "", "", "", "", "", "", "", "" },
},
};
TEST_CASE("tokenizeUrl", "[url][string]")