In a multithreaded, preemptible kernel, there is a need for keeping
out interrupts from certain sections to prevent them from being
interrupted. The main mechanism in Linux, for example, can be
found in
https://github.com/torvalds/linux/blob/master/include/linux/preempt.h
#define preempt_disable() \
do { \
preempt_count_inc(); \
barrier(); \
} while (0)
The counters are kept CPU-local, which can be done relatively easy
on x86 with segment registers. On aarch64, some atomics are needed.
In a multithreaded, preemptible kernel, there is a need for keeping
out interrupts from certain sections to prevent them from being
interrupted. The main mechanism in Linux, for example, can be
found in
https://github.com/torvalds/linux/blob/master/include/linux/preempt.h
#define preempt_disable() \
do { \
preempt_count_inc(); \
barrier(); \
} while (0)
The counters are kept CPU-local, which can be done relatively easy
on x86 with segment registers. On aarch64, some atomics are needed.
Could this be improved, and could hardware help?
In userland, Linux has restartable sequences. Basically, you
tell the kernel that, if a certain code sequence is interrupted,
it should return to a different address than the one it came from.
This code can then recover, retry or whatever. For this, userland
has to hand a data structure to the kernel describing things.
Some details at https://lwn.net/Articles/883104/ .
But the kernel has no such mechanism available.
It could be
possible and useful for the hardware to provide a instruction
modifier
rseq r10,#6
which would be an indirect jump to r10 if there was a hardware
interrupt in the next six instructions.
An alternative could be an instruction modifier
preempt #6
which would schedule all interrupts only after the next six
instructions (with severe limits on the number ant types
instructions as not to lock up the system).
My guess would be that many kernel hackers would be overjoyed at
this functionality.
Comments?
(I did get the second idea from somebody who happens to work as
a Linux kernel hacker at the moment, and who would be overjoyed :-)
Thomas Koenig <tkoenig@netcologne.de> posted:
In a multithreaded, preemptible kernel, there is a need for keeping
out interrupts from certain sections to prevent them from being interrupted. The main mechanism in Linux, for example, can be
found in
https://github.com/torvalds/linux/blob/master/include/linux/preempt.h
#define preempt_disable() \
do { \
preempt_count_inc(); \
barrier(); \
} while (0)
The counters are kept CPU-local, which can be done relatively easy
on x86 with segment registers. On aarch64, some atomics are needed.
Could this be improved, and could hardware help?
It seems to me that rarely is SW in a position to disable all interrupts*, and much more likely to disable (ignore for a while) interrupt below a certain priority. (*) NMI as an interrupt and Machine Check as an exception are more important than the SW critical section at hand.
Secondarily; HyperVisor should be in a position to take exceptions from GuestOS just like GuestOS takes exceptions on behalf of User applications. So, while GuestOS ISR can page fault as long as the PF is handled by HyperVisor and GuestOS ISR does not see any of it happening. We need to
quit thinking about systems with only 2 levels of privilege. --------------------------
My 66000 maintains a priority for CPUs, each thread has its own priority
and each layer in the SW stack has its own priority. This allows one to disallow priorities at lower levels while allowing priorities at higher levels to still interrupt. SW knows about the priority structure, and arranges that higher priority things do not damage lower priority data.
{to hand waving accuracy}.
A thread with priority can directly access its priority in a single instruction; Header Register HR instruction--which has 3 variants:
Read Control Register, Write control register, and Exchange control register.
When transiting the privilege hierarchy, no maintenance of priority
is required--it is done auto-magically--only when SW wants something
outside of it normal operating conditions does SW have to expend any instructions in maintaining priority.
In userland, Linux has restartable sequences. Basically, you
tell the kernel that, if a certain code sequence is interrupted,
it should return to a different address than the one it came from.
This is the underlying premise of My 66000 Exotic Synchronization Mechanism--sometimes you don't want to retry the ATOMIC sequence,
instead, you want to take a fresh look at the concurrent data
structure as it is now {not how it was 1 ms ago}. To the extent
that the OS can plan the ESM-game; it's all FREE.
This code can then recover, retry or whatever. For this, userland
has to hand a data structure to the kernel describing things.
Some details at https://lwn.net/Articles/883104/ .
But the kernel has no such mechanism available.
Kernel could do exactly this same thing with HyperVisor {and then
HV can do the same thing with Secure Monitor.}
Kernel doing this to itself is about as difficult as allowing a
user (application) to take exceptions for itself--which is a non
simple state stacking and unwinding problem (not unlike Delayed
Procedure Calls and the linux variant softIRQ).
Longjump() does not play well with this unwinding...
It could be
possible and useful for the hardware to provide a instruction
modifier
rseq r10,#6
which would be an indirect jump to r10 if there was a hardware
interrupt in the next six instructions.
An alternative could be an instruction modifier
preempt #6
which would schedule all interrupts only after the next six
instructions (with severe limits on the number ant types
instructions as not to lock up the system).
HRX R9,priority,#16 // block lower priority interrupts
...
HRW priority,R9 // back to where we were
Without length limitations.
Would exceptions be allowed during interrupt disablement ??
How does interrupt priority play into taking/ignoring-for-now ?
My guess would be that many kernel hackers would be overjoyed at
this functionality.
Comments?
(I did get the second idea from somebody who happens to work as
a Linux kernel hacker at the moment, and who would be overjoyed :-)
In a multithreaded, preemptible kernel, there is a need for keeping
out interrupts from certain sections to prevent them from being
interrupted. The main mechanism in Linux, for example, can be
found in
https://github.com/torvalds/linux/blob/master/include/linux/preempt.h
#define preempt_disable() \
do { \
preempt_count_inc(); \
barrier(); \
} while (0)
The counters are kept CPU-local, which can be done relatively easy
on x86 with segment registers. On aarch64, some atomics are needed.
Could this be improved, and could hardware help?
In userland, Linux has restartable sequences. Basically, you
tell the kernel that, if a certain code sequence is interrupted,
it should return to a different address than the one it came from.
This code can then recover, retry or whatever. For this, userland
has to hand a data structure to the kernel describing things.
Some details at https://lwn.net/Articles/883104/ .
But the kernel has no such mechanism available. It could be
possible and useful for the hardware to provide a instruction
modifier
rseq r10,#6
which would be an indirect jump to r10 if there was a hardware
interrupt in the next six instructions.
An alternative could be an instruction modifier
preempt #6
which would schedule all interrupts only after the next six
instructions (with severe limits on the number ant types
instructions as not to lock up the system).
My guess would be that many kernel hackers would be overjoyed at
this functionality.
Comments?
(I did get the second idea from somebody who happens to work as
a Linux kernel hacker at the moment, and who would be overjoyed :-)
Thomas Koenig <tkoenig@netcologne.de> posted:
In a multithreaded, preemptible kernel, there is a need for keeping
out interrupts from certain sections to prevent them from being
interrupted. The main mechanism in Linux, for example, can be
found in
https://github.com/torvalds/linux/blob/master/include/linux/preempt.h
#define preempt_disable() \
do { \
preempt_count_inc(); \
barrier(); \
} while (0)
The counters are kept CPU-local, which can be done relatively easy
on x86 with segment registers. On aarch64, some atomics are needed.
Could this be improved, and could hardware help?
It seems to me that rarely is SW in a position to disable all interrupts*, and much more likely to disable (ignore for a while) interrupt below a certain priority. (*) NMI as an interrupt and Machine Check as an exception are more important than the SW critical section at hand.
Secondarily; HyperVisor should be in a position to take exceptions from GuestOS just like GuestOS takes exceptions on behalf of User applications.
In userland, Linux has restartable sequences. Basically, you
tell the kernel that, if a certain code sequence is interrupted,
it should return to a different address than the one it came from.
This is the underlying premise of My 66000 Exotic Synchronization Mechanism--sometimes you don't want to retry the ATOMIC sequence,
instead, you want to take a fresh look at the concurrent data
structure as it is now {not how it was 1 ms ago}. To the extent
that the OS can plan the ESM-game; it's all FREE.
This code can then recover, retry or whatever. For this, userland
has to hand a data structure to the kernel describing things.
Some details at https://lwn.net/Articles/883104/ .
But the kernel has no such mechanism available.
Kernel could do exactly this same thing with HyperVisor {and then
HV can do the same thing with Secure Monitor.}
Kernel doing this to itself is about as difficult as allowing a
user (application) to take exceptions for itself--which is a non
simple state stacking and unwinding problem (not unlike Delayed
Procedure Calls and the linux variant softIRQ).
Longjump() does not play well with this unwinding...
It could be
possible and useful for the hardware to provide a instruction
modifier
rseq r10,#6
which would be an indirect jump to r10 if there was a hardware
interrupt in the next six instructions.
An alternative could be an instruction modifier
preempt #6
which would schedule all interrupts only after the next six
instructions (with severe limits on the number ant types
instructions as not to lock up the system).
HRX R9,priority,#16 // block lower priority interrupts
...
HRW priority,R9 // back to where we were
Without length limitations.
Would exceptions be allowed during interrupt disablement ??
How does interrupt priority play into taking/ignoring-for-now ?
In a multithreaded, preemptible kernel, there is a need for keeping
out interrupts from certain sections to prevent them from being
interrupted. The main mechanism in Linux, for example, can be
found in
https://github.com/torvalds/linux/blob/master/include/linux/preempt.h
#define preempt_disable() \
do { \
preempt_count_inc(); \
barrier(); \
} while (0)
The counters are kept CPU-local, which can be done relatively easy
on x86 with segment registers. On aarch64, some atomics are needed.
Could this be improved, and could hardware help?
In userland, Linux has restartable sequences. Basically, you
tell the kernel that, if a certain code sequence is interrupted,
it should return to a different address than the one it came from.
This code can then recover, retry or whatever. For this, userland
has to hand a data structure to the kernel describing things.
Some details at https://lwn.net/Articles/883104/ .
But the kernel has no such mechanism available. It could be
possible and useful for the hardware to provide a instruction
modifier
rseq r10,#6
which would be an indirect jump to r10 if there was a hardware
interrupt in the next six instructions.
An alternative could be an instruction modifier
preempt #6
which would schedule all interrupts only after the next six
instructions (with severe limits on the number ant types
instructions as not to lock up the system).
My guess would be that many kernel hackers would be overjoyed at
this functionality.
Comments?
(I did get the second idea from somebody who happens to work as
a Linux kernel hacker at the moment, and who would be overjoyed :-)
MitchAlsup <user5857@newsgrouper.org.invalid> schrieb:
Thomas Koenig <tkoenig@netcologne.de> posted:
In a multithreaded, preemptible kernel, there is a need for keeping
out interrupts from certain sections to prevent them from being
interrupted. The main mechanism in Linux, for example, can be
found in
https://github.com/torvalds/linux/blob/master/include/linux/preempt.h
#define preempt_disable() \
do { \
preempt_count_inc(); \
barrier(); \
} while (0)
The counters are kept CPU-local, which can be done relatively easy
on x86 with segment registers. On aarch64, some atomics are needed.
Could this be improved, and could hardware help?
It seems to me that rarely is SW in a position to disable all interrupts*, and much more likely to disable (ignore for a while) interrupt below a certain priority. (*) NMI as an interrupt and Machine Check as an exception are more important than the SW critical section at hand.
Agreed. If the result of an interrupt is a kernel panic or a reboot
of the device, then the integrity of a lock becomes secondary.
Secondarily; HyperVisor should be in a position to take exceptions from GuestOS just like GuestOS takes exceptions on behalf of User applications.
So, Linux (for example) should not be able to run on bare metal?
I think that many people would disagree, Linus Torvalds included.
On 2026-05-19 5:02 p.m., Thomas Koenig wrote:
In a multithreaded, preemptible kernel, there is a need for keeping
out interrupts from certain sections to prevent them from being interrupted. The main mechanism in Linux, for example, can be
found in
https://github.com/torvalds/linux/blob/master/include/linux/preempt.h
#define preempt_disable() \
do { \
preempt_count_inc(); \
barrier(); \
} while (0)
The counters are kept CPU-local, which can be done relatively easy
on x86 with segment registers. On aarch64, some atomics are needed.
Could this be improved, and could hardware help?
In userland, Linux has restartable sequences. Basically, you
tell the kernel that, if a certain code sequence is interrupted,
it should return to a different address than the one it came from.
This code can then recover, retry or whatever. For this, userland
has to hand a data structure to the kernel describing things.
Some details at https://lwn.net/Articles/883104/ .
But the kernel has no such mechanism available. It could be
possible and useful for the hardware to provide a instruction
modifier
rseq r10,#6
which would be an indirect jump to r10 if there was a hardware
interrupt in the next six instructions.
An alternative could be an instruction modifier
preempt #6
I called this the 'atom' modifier for Q+. Disables interrupts except for
NMI for a specified number of instructions. One gotcha is that it can
only apply to instruction groups. So exactly how many instruction are executed with interrupts disabled depends on the mix. The number is a minimum. It is meant only for a small number of instructions.
In My 66000, the compiler puts the instruction-count in the then-clause
and else-clause. Over in the assembler, after the instruction sizes
have been determined, those sizes are changed into word-counts--just like
it determines the offsets for branch labels from the branch instruction.
MitchAlsup <user5857@newsgrouper.org.invalid> schrieb:
In My 66000, the compiler puts the instruction-count in the then-clause
and else-clause. Over in the assembler, after the instruction sizes
have been determined, those sizes are changed into word-counts--just like it determines the offsets for branch labels from the branch instruction.
That is new to me.
The current version of the assembler certainly
does not do so, and we discussed this previously.
When did that change?
Thomas Koenig <tkoenig@netcologne.de> posted:
MitchAlsup <user5857@newsgrouper.org.invalid> schrieb:
In My 66000, the compiler puts the instruction-count in the then-clauseThat is new to me.
and else-clause. Over in the assembler, after the instruction sizes
have been determined, those sizes are changed into word-counts--just like >> > it determines the offsets for branch labels from the branch instruction. >>
We talked about this 2-3 months ago. I remember Brian bringing this up.
MitchAlsup <user5857@newsgrouper.org.invalid> schrieb:
Thomas Koenig <tkoenig@netcologne.de> posted:
MitchAlsup <user5857@newsgrouper.org.invalid> schrieb:
In My 66000, the compiler puts the instruction-count in the then-clause >> > and else-clause. Over in the assembler, after the instruction sizesThat is new to me.
have been determined, those sizes are changed into word-counts--just like
it determines the offsets for branch labels from the branch instruction. >>
We talked about this 2-3 months ago. I remember Brian bringing this up.
Maybe I didn't pay close enough attention at the time... but in
the most recent version of the ISA, it is not clearly specified.
Not sure how to implement this in the compiler/assembler combination
in the presence of instructions whose length can only be determined
by the assembler.
The disassembler needs to re-convert the instruction bytes into
number of instructions (if the TTFF... notation should be kept),
which is impossible without lookahead, and the disassembler cannot
do this (and will not).
I think this needs further discussion.
| Sysop: | DaiTengu |
|---|---|
| Location: | Appleton, WI |
| Users: | 1,118 |
| Nodes: | 10 (0 / 10) |
| Uptime: | 39:22:52 |
| Calls: | 14,340 |
| Files: | 186,357 |
| D/L today: |
23,670 files (7,691M bytes) |
| Messages: | 2,532,986 |