ConcurrentDictionary

bgrommes

I have a use case for a Dictionary potentially shared by multiple client class instances. This raises the possibility that those instances could be used on different threads.

Exposing the .NET ConcurrentDictionary or equivalent collection would be extremely useful here as it would allow multiple threads to performantly add, change or delete dictionary entires without the need for manual external locks.

Both Dictionary and ConcurrentDictionary should implement a common interface, so that it would be possible to use either collection interchangeably at runtime through the interface.

As an aside, I don't love the convention of prefacing all interface definitions with "I" (anymore than I like, e.g., the convention of suffixing all async method names with "Async"), but I can't think of a better way to make it self-evident in your code that a type is an interface. I would like to advocate that built in interfaces follow this convention, at least in cases like this where the sensible interface name collides with one of the sensible class names.

As an alternative, Dictionary could sprout a Boolean IsThreadSafe or UsesLocks or similar named property, which if set to true, could just use the concurrent implementation under the hood (based on the benchmarks I've seen, the concurrent version has a 1.5 to 2.5x performance penalty depending on operation). In that case, the constructor could probably use an optional Boolean argument to select this at time of instantiation. Come to think of it, the property mentioned above should be read-only anyway, at least after the first key is added, as you can't be changing it once you start adding keys.

Garry

Thanks for the suggestion. I’ve looked closely at how this would fit Objo’s concurrency model.

At present, all Objo bytecode within an application VM executes on its owning VM thread. This includes closures scheduled with Task.Run. Native asynchronous operations marshal their Objo-visible results back to that thread, while Worker processes use separate VMs and heaps rather than sharing objects.

Consequently, there is currently no supported situation in which two Objo threads can mutate the same Dictionary simultaneously. A ConcurrentDictionary, common dictionary interface, or thread-safety mode would therefore add substantial complexity without solving a problem that can presently occur.

For that reason, I’m not planning to add this at the moment. If Objo ever introduces shared-memory parallel execution within one VM, concurrent collections would need to be considered as part of a broader design covering atomic operations, enumeration and the safety of stored objects, not simply by exposing the .NET collection.

Your request did expose an error in the documentation though 🙂. It incorrectly stated that Task.Run executes closures on a background thread. I will correct that, and I apologise for the confusion.

Regarding interface naming, Objo’s existing interfaces use names such as Iterable and Comparable rather than an I prefix, so I would retain that convention if a common collection abstraction is introduced later.

bgrommes

Garry Well as far as I'm concerned, that's not only good, it's fantastic. While it's above my pay grade or, honestly, past experience to speculate about whether there's a good use case for shared-memory parallel execution within one VM, my intuition is that it involves lots of additional implementation complexity (like concurrent collections or bespoke locks) in the service of speed. I am a minimalist about concurrency; I don't like to introduce it unless there's a solid reason to. I regard your response as hopeful that there are fewer "gotchas" to consider when doing concurrency with Objo.

I take it this also means that, e.g., Array operations don't need synchronization, as there's no way for two threads to, e.g., InsertAt() at the same time, or to InsertAt() somewhere above where another thread is reading? And so on for any collection? Wouldn't it still be possible for thread A to decide to access index 7 of an array only to find by the time it starts reading it that it has become Index 6 because of a delete on row 3 by some other thread -- or more exactly, to be unknowingly reading the following item because of the delete?

A separate question: in past conversations, you've agreed that Shared getter code does need synchronization to be thread-safe -- can you help me understand whether that's a separate issue or whether your post I'm responding to just addresses the calling of instance methods?

I really don't have comprehensive knowledge or a large body of experience with threading -- I only know that the .NET guidance is to synchronize at least static class field access, including things like incrementing public static counters and the like, even if they don't have getters or setters.

The other thing I know for sure is that trying to anticipate when another thread could intrude and pull the rug out from under you tends to get me, shall we say, out of the flow, to put it nicely :-\

Garry

Yes. Array, Dictionary, and other ordinary Objo collections do not need synchronisation merely because tasks or async code are involved. Within one application VM, there are not two Objo threads simultaneously executing bytecode against the same heap. Workers cannot create that situation either, because each worker has its own VM and heap.

Async code can still make stale assumptions when it suspends. For example:

Async Sub ReadLater(items() As String)
  Var index As Integer = 7
  Await Task.Delay(100.Milliseconds())
  Var item As String = items[index]
End Sub

While ReadLater is suspended at Await, another event handler or continuation could remove item 3. When ReadLater resumes, index 7 might identify a different item or be out of range.

That is cooperative interleaving across a suspension point, not simultaneous collection access by two threads. A concurrent collection would not preserve the meaning of a stored index. Code should revalidate mutable state after Await, or identify the intended item using a stable key or object reference.

You are also right to question my earlier comment about shared getters. A Shared getter written in Objo is still Objo bytecode running on the owning VM thread, and its backing shared fields belong to that VM. It does not require a CriticalSection merely because it is Shared.

The .NET guidance about static fields applies to state that arbitrary .NET threads can access. Native C# code backing an Objo API may require its own synchronisation if it uses process-wide state or background operations, but that is an engine or package-implementation concern. I conflated that with Objo-authored shared state in my earlier answer, so that advice was too broad. Thanks for spotting the inconsistency.

The short version is: ordinary Objo collections and shared properties are protected from thread races by the single-threaded VM model, but mutable assumptions can still become stale whenever an async flow suspends.

bgrommes

Garry Okay so if I understand you correctly, all that's really needed is not to use information gathered prior to an await, after an await, for any operation on a collection, because the possibility exists that the collection mutated in the meantime. You would still not be doing your own operation on the collection at the same time as any other thread, but you might just be making bad assumptions about the collection's current state. Is that an accurate summary?

Also is there EVER a scenario where a critical section in, say, a shared Objo getter would be necessary? I'm having trouble thinking of one if it can only be executed by one Objo thread at a time.

Garry

Yes, with one qualification: you do not have to discard everything learned before an Await. You only need to revalidate assumptions about mutable state that other Objo code could change while the method is suspended.

For example, an array index, count, membership test or value derived from a mutable collection may be stale after an Await. A stable identifier or object reference may still identify the intended item, although the item’s state may have changed. It is safest to treat every Await as a possible yield point, even though an already-completed task may not actually suspend.

There is still no simultaneous collection access. When the method resumes, it gets another uninterrupted turn on the VM thread until it returns or reaches another pending Await. The risk is a stale logical assumption, not a thread race.

For your second question: no, an Objo-written Shared getter that accesses only Objo state in its VM does not need a CriticalSection. Shared fields are per VM, and no other Objo bytecode can interrupt the getter while it runs.

If native C# code uses process-wide state accessible from several .NET threads or VMs, that native implementation must perform its own synchronisation. An Objo CriticalSection in the getter would not protect state shared across separate VMs.

More generally, a CriticalSection is unnecessary for an ordinary synchronous Objo state change, and it cannot protect a workflow across Await because Await is forbidden inside a Lock. For async coordination, use a Boolean guard, generation counter or single-consumer queue, depending on the intended behaviour.

I’ve clarified this distinction in the async/await documentation for the next release.

bgrommes

Garry Thanks for your careful and patient answers. I appreciate it.

bgrommes

@Garry I am trying to develop correct habits of thought about design in Objo code such that any code I write would be thread safe even if I'm not particularly contemplating an async use case. "Thread safe" in the loose sense of "potentially reentrant by other execution contexts caused by resuming awaits". I understand that technically there is no thread.

I would just like to clarify a point here. Is it true that the VM could give control to another "thread" in between any two statements? Such that while, say, one call to a non-Objo method to mutate a collection would wait to return before allowing another call from a different context that happens to resume at the same spot in the code, if you had TWO mutation calls that need to be atomic, e.g., you are deleting one item from an Array and re-inserting it elsewhere, it is possible for another execution context to be active in between those calls, and therefore, the deletion and insertion should be in a critical section? Or if you have a call to a method that's Objo code that performs such multiple operations that should guarantee atomicity, that method should consider its internals to be a critical section, because that method itself could be interrupted between any two statements it's executing?

I'm using the term "critical section" generically here to mean either a literal critical section or a Boolean guard variable or some other more economical construct that might be sufficient to the task (how THOSE decisions are made might be a separate topic).

What I find compelling here is that compared to .NET, you have succeeded via the single threaded VM in automatically constraining how async/await/tasks do their thing, such that it is relatively far easier to reason about and to routinely produce await/task-safe code without going down rabbit-holes of seemingly infinite possibilities. It's still extra cognitive effort, but not so much that I would throw my hands up, document something I'm building as "not thread safe" and leave it as a problem for the client code. In other words it becomes reasonable in the service of modern hardware and software, to opt in to playing this particular game, if you will.

Garry

@bgrommes: Your question exposed an important discrepancy between Objo’s intended concurrency model and what the VM was actually doing.

My earlier answers described the intended behaviour: synchronous Objo code should run uninterrupted until it returns control to the scheduler or reaches an Await that genuinely suspends. After reproducing the exact scenario you described, however, I discovered that the VM was processing queued task callbacks, timers, host events and desktop events between individual bytecode instructions.

It wasn't running two Objo threads simultaneously, but another execution context could effectively re-enter the VM between two ordinary statements. Therefore, in the current release, another task or event could observe the intermediate state between removing an array item and reinserting it.

A CriticalSection did not reliably solve this because the re-entry occurred on the same VM thread and the underlying lock was re-entrant. Thank you for pressing this point: my previous explanation was correct as a design goal, but not as a description of the implementation.

I've now corrected the scheduler comprehensively. This was a royal pain in the arse to fix! In the next release:

Synchronous Objo code gets an uninterrupted turn.
Task.Run, task completions, timers and queued native events wait for a scheduler boundary.
Multiple collection mutations within one synchronous method cannot be interleaved by another task or event.
Task.Yield now genuinely suspends execution until a later scheduler turn.
A pending Await remains a yield point, so assumptions about mutable state must still be revalidated after it.
Queued UI control events obey the same rule, while preserving the correct event context: Me is the event source and Self is the subscriber.
Explicitly nested event loops created by Window.ShowModal remain the documented exception, because showing a modal window must continue processing events.

Consequently, ordinary synchronous operations on an Array, Dictionary, shared field or other Objo state do not need a CriticalSection merely to prevent task or event re-entry. If a method removes an item and reinserts it before returning or awaiting, that whole sequence is now one uninterrupted turn.

I've also strengthened shutdown, exception propagation, debugger stopping and object-lifetime handling around these scheduler changes, and updated the async, task, critical-section and window documentation.

My internal benchmarks show no material VM performance regression. The per-instruction hot path remains essentially unchanged, with the new scheduling work occurring only at turn boundaries.

This will be included in the next release: https://feedback.objo.dev/bug/1364

Thanks again for questioning this so carefully. This was a real VM bug, and your example was what led me to find and correct it.

bgrommes

Garry This was a royal pain in the arse to fix!

Well I didn't think I was going to set off a chain reaction of hair pulling but I'm glad you were able to get it working as intended!

I'm a little fuzzy on the definition of the "uninterrupted turn" that synchronous Objo code gets. In an ideal world, any given synchronous method I find myself contemplating should get an uninterrupted turn to do all its work, and that's what you're saying I think -- but any one method can run for arbitrary amounts of time and it's called by other code that's called by other code potentially many levels deep. What is the "unit of work" that can't be interrupted, and maybe even more relevant, what is the unit of work that CAN be interrupted?

Maybe the relevant rule for me when I'm writing an synchronous method, is that at least until I Return from that, I have it to myself?

I'm not 100% sure I'm being clear or even asking a coherent question, lol. I am running on too little sleep and too much distraction at the moment. I may need to think through some scenarios when I have a little more time, but if you are able to address the above, it'd be a start anyway.

Garry

Your question is perfectly coherent. The key distinction is that a turn is not one method call: it is the entire synchronous call stack started by one scheduler-dispatched action.

For example, if a button event calls MethodA, which calls MethodB, which calls MethodC, all three methods are part of the same turn. Returning from MethodC to MethodB, or from MethodB to MethodA, does not allow queued tasks or events to run. The turn ends only when the outermost event handler, task closure, continuation or application entry point returns control to the scheduler.

So your proposed rule (“until I return from my synchronous method, I have it to myself”) is correct, but the actual guarantee is slightly stronger. If your method returns to another ordinary synchronous Objo method, that caller continues uninterrupted as part of the same turn.

There is no arbitrary statement, method-call or time-based pre-emption. Even if a synchronous method runs for several seconds and calls through many levels of other synchronous methods, queued tasks, timers and UI events must wait. The downside is that such a method also prevents the application from processing those things, so lengthy work should deliberately yield when responsiveness matters.

A new turn can begin when the current work returns to the scheduler or suspends at a genuine yield point. In practice:

An Await of an incomplete task suspends that async flow and permits other work to run.
Task.Yield always yields to a later scheduler turn.
An Await of an already-completed task continues inline and does not end the turn.
Scheduling something with Task.Run does not itself yield; the closure waits until the current turn finishes.
Window.ShowModal is an explicit exception because it deliberately runs a nested event loop while its caller is paused.
A user-written RaiseEvent is synchronous. Its handlers execute immediately as part of the current turn, just like calling another method, rather than arriving unpredictably from the scheduler.

Therefore, two consecutive collection mutations in ordinary synchronous code cannot have a queued task or UI event inserted between them. Code after a pending Await, however, executes in a later turn and must treat mutable state as potentially changed.

A compact way to think about it is: Objo never takes the VM away from synchronous code unexpectedly. Your code keeps the VM until the current outer turn finishes or explicitly reaches something that yields.

bgrommes

Garry lengthy work should deliberately yield when responsiveness matters.

Is there a way for a synchronous method to do that? Or would that be the time to refactor it to be asynchronous?

I guess that's another way of asking -- is there any way to "explicitly reach something that yields" in synchronous code "before the outer turn finishes"? Or is yielding a thing that only async code can do?

As a fine point -- since the entry point is implicitly async (IIRC for both console and GUI apps), I presume that it is not exactly a question of whether a method is async as to whether your code is out of a protected synchronous context, but whether it is async and has at least one await within its code. Is that accurate?

Garry

Yes:. An ordinary synchronous method cannot voluntarily yield and later resume. Calling Task.Yield() without Await does not yield. If lengthy work must remain responsive, the controlling method should be async and periodically use:

Await Task.Yield()

You need not make every helper async. A useful pattern is an async driver which performs bounded chunks of synchronous work and yields between them. Each chunk remains uninterrupted; between chunks, the scheduler can process pending UI events, timers, tasks and continuations. Task.Yield() does not move computation to another thread or run it in parallel; a Worker is the better choice for independent CPU-intensive work.

Your fine point is close, with two important qualifications:

Merely declaring a method Async, or even having an Await somewhere in its body, does not itself end a turn.
A turn ends only when execution reaches an Await whose task is still pending. Awaiting an already-completed task continues inline. Await Task.Yield() is the convenient way to guarantee suspension until a later scheduler turn.

An async method therefore never becomes generally “unprotected”. It is divided into uninterrupted synchronous segments separated by genuine suspension points. Other work may run while a segment is suspended, so the next segment must revalidate assumptions about mutable state; once that segment starts, however, it again runs uninterrupted until the next genuine suspension or the outer turn finishes.

One correction regarding entry points: CommandLineApplication.Run and WorkerApplication.Run are implicitly async-capable, so they may use Await without an Async modifier. A desktop application starts through Opening(args() As String), which must be declared explicitly as async if it needs to await:

Async Event Opening(args() As String)
  Await Task.Yield()
End Event

bgrommes

Garry OK great, this is super helpful and serves as solid first principles that I can build on. Importantly, any exceptions are also clearly delineated. Thanks once again!

I will doubtless have questions over time but this will definitely get me started with the confidence that I'm not working with bad assumptions.

My guess is that the standard prompt for the AI assistant will do well to have the same knowledge / ground rules, unless the new LLM-friendly docs will adequately delineate it or smaller additions to the prompt can point to the relevant doc section(s).