Ability to pre-initialize Arrays

bgrommes

In porting C# code to Objo, I'm encountering situations where in C# I used actual fixed-length typed arrays. Sometimes this is the better metaphor: fixed-size buffers, short lists of values that might not be assigned in ordinal order, etc. Objo arrays are really List<T>s under the hood, which takes a smidgen of getting used to but is fine for most situations.

However sometimes I do want an Array(Of T) to be pre-initialized to a known size and pre-populated with default values (zero for numerics, Nothing for reference types, DateTime.MinValue for dates, etc) so that I can then very rapidly read or write individual values by index without worrying about whether there's anything there in the first place.

I thought about several possible approaches to this issue and to my mind the cleanest / simplest solution that would cover most of the use cases I can anticipate, would be an optional count argument to the constructor:

Var foo = new Array(Of Integer)(10)

This would set capacity to 10, populate indexes 0-9, and set Count of course to 10. After that, one uses the Array however one wishes, including potentially making Append() calls later. But ordinarily you would do this so you could just address individual rows for reads/writes by index, knowing that the size is pre-initialized to what you need.

At present I have an ArrayHelper class that prepopulates an array for me but for the above example it has to do 10 Append() calls in a loop, which is awkward / inconvenient for the use case and probably significantly slower than the VM doing or delegating to the .NET runtime to do it. This becomes more of a concern with large arrays. My largest so far is 8,192 elements, although only initialized once per instance and so not a hot path concern. Later, that might not be the case so much.

Instead of the constructor argument (or IMO, in addition to it) you could have a Resize() method with a couple of overloads that would provide maximum flexibility and power. It would work as follows:

Var foo As New Array(Of Integer)
foo.Resize(10) # same as the optional constructor in this situation, but could be called on a non-empty array
# or:
foo.Resize(10,42) # initialize all elements to something other than the type default (zero in this case). This is arguably the least important new functionality, but it covers all the bases.

The rules would be:

If the requested # of elements is > current Count property, add those elements to whatever is already there. If Capacity is a smaller number, Reserve(Count) first. Only new elements are initialized.
If the requested # of elements is < current Count property, truncate to that size. Leave Capacity alone; the caller can always TrimExcess() if they feel the need.
If the requested # of elements is = current Count property, it's a no-op.

The argument against a Resize() method is that it makes the API for Array(Of T) too fussy or confusing for some users. Although I think it's pretty clear what you would use it for, maybe it's too similar to ReserveCapacity() and you'd get bogged down in explaining the difference (though I don't personally buy that argument). Functionally, my helper method already does this, but again, if you're making a big increase or decrease in size you are faced with repeated Append() or RemoveAt() calls, and that is slower.

I supposed that Resize() could also have a boolean to specify a forced overwrite of all elements with the default or specified value, so you have a clean slate. But one could also split the responsibilities into two methods, like Resize() and ReInitialize(). Yet Another Method, but also maybe clearer / easier to document and explain.

I spitballed other possibilities too -- e.g., make Count writable, or have a separate FixedArray(Of T) class. IMO the optional constructor argument on the existing Array(Of T) is simple and clean and easy to explain, Resize() (or Resize() and ReInitialize()) would be nice to have, and all that is a smaller blast radius than the alternatives.

Garry

Thanks for the thoughtful and detailed suggestion. You’ve identified an important distinction between reserving capacity and creating addressable elements with a known initial value.

Array already supports most of the proposed behaviour through two Resize overloads:

Var foo As New Array(Of Integer)
foo.Resize(10)      # Count becomes 10; new elements contain 0
foo.Resize(20, 42)  # New elements contain 42

When growing an array, Resize(count) uses the element type’s normal default value. When shrinking, it removes elements from the end; resizing to the current count does nothing. This work happens natively, so it avoids making repeated Objo-level Append() calls.

I'm not totally against the constructor syntax you proposed:

Var foo As New Array(Of Integer)(10)

Possibly accompanied by:

Var foo As New Array(Of Integer)(10, 42)

These could be convenient creation-time equivalents of the existing Resize overloads, without introducing another kind of collection but they aren't particularly Objo-ish (although that is of course very subjective!).

I’m less keen on making Count writable or adding a Boolean that changes the meaning of Resize, as those behaviours are less explicit. Likewise, a separate FixedArray would add considerable API surface when the existing dynamic array can already support this use case.

bgrommes

Garry Well I guess I missed Resize()! That's the ticket, as far as circumventing repeated Appends().

That just leaves the convenience constructor syntax, which is super nice if you know up front what you want, which is usually the case. So this:

Var foo As new Array(Of String)(100)

... is way more concise than:

Var foo As New Array(Of String)
foo.Resize(100)

The latter is like you are changing your mind, while the first shows your actual intent up front -- plus it's one call and not 2, I'm sure a little bit of overhead gets avoided that way, and not just call overhead.

Yes my other ideas were just cycled through and discarded; I only personally liked the constructor and the ability to adjust later if needed. Just adding the constructor(s) with count and optional value would be sufficient to cover both situations, given the presence of Resize(). Next time I need to pay more attention to the docs rather than blue-skying duplicate features. In my defense, out of habit, I tend to think of Objo arrays as List<T>s and those don't have pre-initialization other than indirectly via newing up an array and passing that to the constructor.

Ormente

bgrommes I would like this:

Var foo As new Array(Of 100 String)

even more! But I'm admitedly not a Basic purist 😁

Garry

I like the constructor ideas, and I’ve now implemented it with the following syntax:

Var names As New Array(Of String)(100)
Var flags As New Array(Of Boolean)(100, True)

It will be in the next release: https://feedback.objo.dev/feature/1160

I decided to keep the size in the constructor arguments rather than use Array(Of 100 String).

The main reason is that Of denotes generic type arguments in Objo. String is part of the array’s type, whereas 100 is a runtime value that could also be a variable or expression. Putting the size inside Of might also imply a new fixed-size array type, but these arrays remain fully dynamic.

New Array(Of T)(count) creates an array with an initial Count of count, filled with the normal default value for T. For example, integers receive 0, strings receive "", reference types receive Nothing, and nested arrays receive independent empty arrays.

New Array(Of T)(count, defaultValue) fills every element with the supplied value. If it is an object reference, each element initially refers to the same object.

The resulting array can still be resized and appended to normally. This is convenient initialisation rather than a separate fixed-size array type.

This required changes across the type checker, compiler and VM rather than being purely syntactic sugar, but the resulting API is small and feels consistent with the rest of Objo.

Thanks for the suggestion!