When constructing strings in C, the
snprintf() and
vsnprintf() are one of the first tools we reach for. Unfortunately, in many embedded and freestanding environments they may not be available or their code size is too large to be practical; and even in hosted POSIXy systems, one cannot use this in signal handlers because they are not
async-signal safe.
I recently realized that the tools I usually reach for in these situations can be divided into two categories:
small string buffers and
reverse builders. Small string buffers are exactly what they sound like, just written to be small and efficient (both time and space) with minimal to no external dependencies (so usable even in interrupt handlers in embedded environments, and in signal handlers in hosted POSIX C enviroments). Reverse builders are also string buffers, except they work bass-ackwards, from the end towards the beginning of the string. If the string consists (mostly) of integers in decimal form, the reverse builders are desirable as they don't do any extra work at all (like moving the string buffer around, or extra divisions by ten to find out how many characters one needs to reserve for them; most integer formatting implementations tend to have to do one or the other).
So, let me describe the
small string buffers: their use cases, their design requirements, and my preferred implementation. If you are a new C programmer or still learning about these things, perhaps there is something about the process or how one ends up with this kind of thing that may be useful to you. If you are an old hand at C (or the subset of C++ used in embedded environments, which sadly usually omit all the nice C++ string stuff), perhaps you have a different approach you use, that you care to contrast and compare to? Or if you find this yukky, feel free to say so, although I do need to point out I only have these because I've needed them in practice before. You cannot always reasonably punt the work from the constrained/limited/critical section to say a worker thread; sometimes it just makes most sense to do it right there in the bind, even if one needs to implement ones own tools for it.
Use casesMost typical use cases are human-readable time stamps ("
YYYY-MM-DD HH:MM:SS.sss" being my preferred format) and reporting variable values to a buffer or stream of some sort, for logging purposes, or as responses to a query or a command.
The common theme here is a very restricted environment. Your code may be an interrupt handler on an AVR, with limited memory and computing resources, and because almost none of the base library functions are re-entrant (safe to use in an interrupt handler), you either do the formatting elsewhere, or with your own code. Or you might have a Linux program or daemon of some sort, and want to log some signals directly from their signal handlers, without setting up a thread to sigwait()/sigtimedwait() on those signals so one is not restricted to async-signal safe functions. If you can use snprintf()/vsnprintf(), by all means do so: you should only reach for this kind of alternative when you reasonably cannot use snprintf()/vsnprintf() or whatever the formatting functions provided by your C (or subset-of-C++) environment provides.
For the same reasons, these strings are severely restricted in length. Typically, they are less than a hundred characters long. Sometimes you have a small formatted part before and/or after a bulk data part, so the total may be much higher. I personally find a hard upper limit of say 250 characters here perfectly acceptable.
When working on embedded systems, I do not want to waste RAM, though. For the same reason, I don't want to use C++ templates either for each different string buffer length; I need the functionality small and tight – think very specific surgical tool, not a swiss army knife.
Design criteriaI want to specify the string buffer size at compile time (or if on-stack, at run time), and not waste any memory.
I want a single set of functions to act on the buffers. I do not want multiple copies of the functions just because they access string buffers of slightly different sizes.
I want the functions to be fully re-entrant (meaning, if code working on one string buffer is interrupted, the interruptee is safe to work on any other string buffers) and async-signal safe.
I accept a low hard upper limit on the buffer size, say around 250 chars or so. Typically, mine are much smaller, on the order of 30 or so.
I want these to be efficient. They don't need to be
optimized yet, just do what they do without wasting effort on theoretical stuff.
I want each string buffer to also record its state, so that I can stuff the components that form the strings into it, and only have to check at the end whether it all fits.
Practical use exampleAn interrupt or signal handler needs to provide a string containing a timestamp in human readable form. The timestamp is a 32-bit unsigned integer in milliseconds. That is, if it has value 212×1000×60×60+34×1000×60+56×1000+789 = 765,296,789 = 0x2D9D8095, the timestamp should read "
212:34:56.789". Thus:
uint32_t millisecs; // To be described in human readable form SSB_DEFINE(timestamp, 14); // Maximum 32-bit timestamp corresponds to 1193:02:47.295, which has 14 characters const int hours = millisecs / (60*60*1000); millisecs -= (uint32_t)hours * (60*60*1000); // or millisecs %= 60*60*1000; ssb_append_int(timestamp, hours, 0); ssb_append_chr(timestamp, ':'); const int minutes = millisecs / (60*1000); millisecs -= (uint32_t)minutes * (60*1000); // or millisecs %= 60*1000; ssb_append_int(timestamp, minutes, 2); ssb_append_chr(timestamp, ':'); const int seconds = millisecs / 1000; millisecs -= (uint32_t)seconds * 1000; // or millisecs %= 1000; ssb_append_int(timestamp, seconds, 2); ssb_append_chr(timestamp, '.'); ssb_append_int(timestamp, (int)millisecs, 3); if (ssb_valid(timestamp)) { // string is ssb_ptr(timestamp, NULL), length ssb_len(timestamp, 0). } else { // Failed. Either buffer was too short, or a conversion couldn't fit // in the allotted space, or something similar. }In a file scope, one can declare a small string buffer via say
SSB_DECLARE(common, 30);and in each scope where it is used –– but note that these uses must not interrupt each other! ––, it is just initialized with
SSB_INIT(common);In a local scope, to reuse memory currently unused for a small string buffer, one can use say
SSB_STEAL(newname, pointer_to_memory, size_of_memory);but again, it is up to you to make sure that while this memory is being used as a string buffer, we don't get interrupted by some other code that also uses it. Similarly, if we steal it, we'll be rewriting its contents, so at this point it better not have any useful stuff in it anymore.
This will declare a new variable named
newname, compatible with the ssb_ functions, pointing to the data. Hopefully, the compiler is smart enough to optimize the variable away. (In C, one can only make the new one
const, but in C++ the
newname pointer can be a
constexpr.)
To append floating point number in decimal format (not scientific format) using six decimal places, one could use say
ssb_append_float(buffer, value, 0, 6);where the first 0 indicates "as many digits as needed on the left side of the decimal point", and 6 "six digits on the right side of the decimal point".
(I like using positive numbers to indicate fixed numbers of digits with zero padding, negative for fixed width with zero padding, and zero for "the size of the value without any extra padding". I do not normally have a way to force a positive sign; the decimal strings that I tend to need are either negative (with a negative sign), or nonnegative (without any signs).)
In hosted environments, I like having
ssb_write(buffer, descriptor); which tries hard to write the string buffer contents to the specified descriptor. That is, if the string buffer is valid, and none of the operations on it have failed; if they have, then this (and all other ssb_ functions) do nothing.
ImplementationA small string buffer is an array of 4 to 258
unsigned chars.
The first one is the maximum number of chars it can hold (
maxlen), not including the trailing nul char. This one must be between 1 and 254 (
UCHAR_MAX-1), inclusive; the smallest and largest possible values are reserved (and detected as *broken* buffers, scribbled over by some other code).
The second one is the current number of chars in the buffer (
len), not including the trailing nul char.
There is always room for the trailing nul, but an implementation may only set it there when the buffer pointer is obtained via a
ssb_ptr() call.
Each small string buffer has three states:
- Valid: Operations have all succeeded thus far. (maxlen >= 1 && maxlen < UCHAR_MAX && len >= 0 && len <= maxlen).
- Invalid: Operations have failed. (maxlen >= 1 && maxlen < UCHAR_MAX && len > maxlen).
- Broken: Someone overwrote maxlen. (maxlen < 1 || maxlen >= UCHAR_MAX).
Initializing operations cause the buffer to have
maxlen computed based on their size, with
len zero, and the first byte of data also zero. The rest of the data bytes do not need to be initialized. (On embedded architectures, this uses three assignments instead of an initializer, to keep the ROM/Flash overhead minimal.)
Aside from the maximum length value, and the state logic encoded by the length and maximum length values, this is a common string or string buffer format, called either length-prefixed, or more commonly Pascal strings, since many Pascal compilers use(d) a length-prefixed format for strings.
ssb_valid(),
ssb_invalid(), and
ssb_broken() functions can be used to check the buffer state, but I rarely need them; usually only to set up a debugging flag if a report had to be skipped because the string buffer was too small. These functions also tend to be
static inline and/or have the
pure used leaf function attributes, so that the compiler doesn't include them at all in the final binary if they're not used, can can optimize them with rather extreme prejudice. They are so simple that even on AVR, it would be more effort to move the data to suitable registers for a function call, than just straight-up inlining the accesses involves.
Other
ssb_function() only operate on small string buffers in the
Valid state. If the buffer is in
Invalid or
Broken state, it is not modified nor accessed beyond the two bytes that define
maxlen and
len.
A particular detail I like is that the pointer accessor function takes an extra
ifbad argument,
char *ssb_ptr(buffer, char *ifbad);as does the length accessor function,
int ssb_len(buffer, int ifbad);so that having e.g. ternary expressions, one can just handle the invalid/broken buffer states with a custom return value.
These are static inline accessor functions, because on most architectures moving the parameters to their respective registers is as much code as doing the accesses, so the extra
if-bad parameter/return value doesn't really cost anything.
My current implementations for the equivalents of
ssb_append_int() and
ssb_append_float() do include an "extra" loop that divides the integral part by ten repeatedly, to find the number of integer digits needed. That loop is then repeated to generate the actual digits. (This can be avoided by using buffer space left starting at the very end, so that only one divide-by-ten loop is needed, followed by copying/moving the data to the correct position in the buffer. Both have their use cases.) The fractional digits are generated by repeated multiplications by ten, followed by extracting the integer part, so is also suitable for any fixed-point formats one might use (although one does need to write the
ssb_append_type() function for each different fixed-point format).
While the repeated division-by-ten with remainder giving the next decimal digit, and multiplying fractional part repeatedly by ten and grabbing the integral part for the next digit (and incrementing it if necessary for the final digit after comparing the final fractional part to 0.5), may
sound inefficient, they actually easily outperform most
printf() implementation variants with the same formatting specs, while producing the same string. So, I definitely do not consider this (or at least mine) small string buffers
optimized, I do consider them
efficient. With possible exception being that integer part loop on different architectures: those that have a fast and compact integer divide-by-ten-and-keep-remainder machine instruction will prefer the extra division loop, others will prefer copying the few bytes over.
I can provide my own implementations as CC-BY, but at this point, I haven't fully integrated everything into a single codebase, and I'm seeing so much
#ifdef macros in the single-source, I think it might be better to keep the interface same but decidedly different implementations (say, AVR vs. AMD64) separate. I'm not sure about whatchamacallit, either; my source right now uses
tinystr, but I think some three-letter acronym implying "buf" would work better, although "ssbuf" sounded a bit off to me to use. And my idea here was not to look at just
my code, I want to see what others do and use. You know, shop talk.
Those I've talked to face-to-face tell me they don't like showing this kind of code, because although theirs is just as functional as mine, this kind of constrained code is often left in its ugliest working form; only worked on if it breaks, rewritten from scratch (or from memory) for each use case. Because face it, people most often just choose to wing it instead, hoping that
this time they won't get bitten by re-entrancy issues or async-signal unsafety, if they use snprintf() etc.; and if they do get bitten, they (and I myself) more often punt these to worker threads and less constrained parts, rather than work on such ugly little code snippets. In all honesty, just look at even the outlined interfaces: I even need macros to declare/initialize/define the buffer variables, and that just starts the odd side.
The reason I originally started paying attention to integer and floating-point number formatting, was when I was dealing with huge molecular datasets ("movies", dozens of gigabytes in slowish spinning rust, well over a decade ago now) in a nicely spatially and temporally sliceable distributed binary form (dataset scattered over hundreds of binary-formatted files). I needed to generate the interchange/transport formats (various human-readable text formats for visualization et cetera), and that severely bottlenecking on printf(). A main source saving to those binary files was written in Fortran 95, too... So, I wrote faster convert-doubles-to-string versions, and verified the strings matched for all normal (as in finite non-subnormal) doubles. (Well, not all 2
62 or so of such values, but maybe 2
40 of them.) It was so ugly I hid the code, but it was/is functional.