Integer32 and UInteger32 would benefit from some performance tuning

bgrommes

I've done some benchmarks on counting to 1M in an empty loop for Integer vs Integer32 vs UInteger32:

Runtime backend: vm
1M iterations with Integer: 0.035
1M iterations with Integer32: 0.391
1M iterations with UInteger32: 0.406

I doesn't surprise me that the new types aren't necessarily faster, but I did not expect 10-12x slower.

Appending to an array of the same type in a loop with 1M iterations is a little less divergent:

Runtime backend: vm
Create 1M Array(Of Integer): 0.307
Create 1M Array(Of Integer32): 0.850
Create 1M Array(Of UInteger32): 0.851

Finally, creating 1M (U)Integer32 elements but while using an Integer for the array loop counter is less divergent still:

Runtime backend: vm
Create 1M Array(Of Integer32) using Integer counter: 0.551
Create 1M Array(Of UInteger32) using Integer counter: 0.552

These are all after warmup.

Now Garry this isn't a hill I want to die on but I'm just putting it out there. I know you added these because a plugin vendor was dealing with these 32 bit types in existing data and the expectation that they would have a particular wrapping behavior. And you have advised that people should still default to Integer for most everything. I am simply exploring some possible edge case uses. In particular if I needed, say, a hundred thousand or more pointers in memory none of which really need the range of an Int32 much less an Int64, being a little less extravagant with memory would be nice.

A few of my thoughts:

1) The new types are not bounds checked by design which makes them dangerous to use in a way, if you're unaware of that limitation. E.g., Integer32.MaxValue += 1 does not result in an overflow error by design, but results in Integer32.MinValue. So people need to be aware of that.

2) It is not a matter of simplistically cutting the storage of each element from 8 to 4 bytes, it is more like from 32 to 28 bytes because of the slot overhead on the VM side of things. There is still a 4 byte savings per element on the .NET side (correct me if I'm wrong). The point being, space savings isn't as compelling and depending on the app it's maybe not even something that's at a premium. It would only come into play for very large arrays, particularly if they are around for the life of the program and maybe particularly if they are static once loaded.

So ... at any rate. What I am suggesting is that if it's not a lot of work / ongoing maintenance for at least Integer32 to enjoy some or all of the VM optimizations that Integer clearly does, that'd be great. Failing that I still see a use case for very large arrays as described above and I would say that while the perf tradeoff would be worth it to me, it'd be nice if it were more of a no-brainer because the diff between Integer and Integer32 arrays could be narrowed or eliminated.

I also am not saying this should be high priority. Just advocating for it.

Thanks.

Garry

Thanks for benchmarking this. I agree that a gap of that size is worth investigating.

There is an optimisation gap in the current VM: ordinary Integer For loops with an implicit step use specialised instructions for checking and incrementing the counter. Integer32 and UInteger32 loops currently use a longer sequence, including conversions on each iteration. An empty loop makes that overhead particularly visible.

That is a likely contributor to your results, although I need to reproduce the benchmarks and profile them before attributing the whole difference to it. Any optimisation also needs to preserve correct loop termination at the 32-bit limits.

One correction on memory usage: the 32-bit types currently provide no per-element memory saving in arrays. All three integer types occupy the same 24-byte runtime value slot. An Array(Of Integer32) therefore uses the same element storage as an Array(Of Integer). The compiled backend also uses this representation, so there is no automatic four-byte saving per element there either.

Compact four-byte array elements would require a separate packed-array implementation. The current benefit of these types is their specific range, signedness and wrapping arithmetic.

I’ve logged this for investigation, including the array timings. Having your exact benchmark source would help me separate counter overhead from element conversion and append costs.

For now, I would still recommend Integer for general counters and indexes, and the 32-bit types where their arithmetic behaviour is needed.

bgrommes

Thanks, Garry!

Here you go ...

Pragma Unused args
Var bm As New Benchmark()
bm.StartBenchmark("1M iterations with Integer")

For i As Integer = 1 To 1000000
Next

bm.EndBenchmark()
bm.StartBenchmark("1M iterations with Integer32")

For i As Integer32 = 1 To 1000000
Next

bm.EndBenchmark()
bm.StartBenchmark("1M iterations with UInteger32")

For i As UInteger32 = 1 To 1000000
Next

bm.EndBenchmark()

Var integerArray As New Array(Of Integer)
bm.StartBenchmark("Create 1M Array(Of Integer)")

For i As Integer = 1 To 1000000
  integerArray.Append(i)
Next

bm.EndBenchmark()
Var integer32Array As New Array(Of Integer32)
bm.StartBenchmark("Create 1M Array(Of Integer32)")

For i As Integer32 = 1 To 1000000
  integer32Array.Append(i)
Next

bm.EndBenchmark()
Var uInteger32Array As New Array(Of UInteger32)
bm.StartBenchmark("Create 1M Array(Of UInteger32)")

For i As UInteger32 = 1 To 1000000
  uInteger32Array.Append(i)
Next

bm.EndBenchmark()
integer32Array.Clear()
bm.StartBenchmark("Create 1M Array(Of Integer32) using Integer counter")

For i As Integer = 1 To 1000000
  integer32Array.Append(Integer32(i))
Next

bm.EndBenchmark()
uInteger32Array.Clear()
bm.StartBenchmark("Create 1M Array(Of UInteger32) using Integer counter")

For i As Integer = 1 To 1000000
  uInteger32Array.Append(UInteger32(i))
Next

bm.EndBenchmark()
bm.PrintResults()

Garry

Thanks, that gives me the loop and array code I needed. These are exactly the implicit-step For loops where the optimisation gap I described applies.

There is one detail to account for in the array comparisons: Clear() removes the elements but retains the allocated capacity. Your final two tests therefore refill arrays that already have room for a million elements, whereas the earlier tests include growing the backing storage. The lower timings can reflect both the different counter path and avoiding that growth.

To separate those effects, I would call Reserve(1000000) on every array before starting its timer, including the Integer array. Fresh-array filling can then be measured separately, using a fresh array for every case.

The final two tests also perform an explicit checked conversion on each iteration through Integer32(i) or UInteger32(i). Their timings include that conversion work alongside appending, so I will profile those conversions as well as the loop bookkeeping.

The empty-loop comparison remains useful and is unaffected by the array-capacity difference.

I’ve added your code and these findings to the investigation. Could you also share the Benchmark class and the Studio version you used? That will help me match your timing and warm-up setup.

bgrommes

Garry I used 26.11.3 (MacOS).

Benchmark has other dependencies. For this use case it is basically just subtracting System.Ticks before & after, though.

Is there a way to share short of doing a Save As on an entire solution and then deleting everything that doesn't apply?