String Interning

bgrommes

In .NET there is a string interning system, normally used only for string constants in code and a few other special cases (e.g., String.Empty or "" is a singleton string). The basic idea is that every string constant exists only once in memory.
This can be tapped into when you want to trade off a bit of loading performance for memory savings. For example if you're loading business names and addresses into a collection, it's a huge memory savings to intern the state abbreviation, and depending, perhaps the postal code and city names. If you have 1,000 US addresses, you would potentially have perhaps at most 50 unique state code strings in memory rather than 1,000. Little things like this add up.
It's just a matter of assigning like this:
string x = string.Intern("AZ");

The downside as I mentioned is that's a slower assignment since it involves some sort of hash or dictionary lookup, and also, "AZ" is in the string interning pool until the app quits -- there's no way to prune it at runtime.

My guess is that it would not be a lot of work to expose the interning API to Objo and I would definitely use it.

(Possibly this should be under "suggest a feature" but here again I am not sure whether Objo rolls its own UTF8 string support or taps into some of the .NET internals. Also -- it's likely that strings would be interned as UTF16. If so, for strings with lots of characters in the ASCII range that would dilute some but by no means most of the memory savings possible in Objo).

Garry

Thanks for the suggestion. Objo strings are indeed backed by .NET strings, so exposing interning is a good fit here.

I’ve added String.Intern() for the next release (https://feedback.objo.dev/feature/434). It returns the exact string value from the process-wide intern pool, so this will work well for bounded repeated values like state codes, status names, categories, etc.

One important detail: Intern() does not normalise the string. So "az".Intern() and "AZ".Intern() are separate interned strings, even though Objo string comparisons are case-insensitive by default. If you want canonical storage, normalise explicitly first:

state = state.Trim().Uppercase().Intern()

I’ve also documented the usual caveat: interned strings remain in the intern pool for the lifetime of the app, so this should be used for bounded vocabularies rather than free text, GUIDs, timestamps, URLs, or other mostly-unique values.