Repository navigation
Updates - #64
Merged
Merged
Updates#64
Conversation
Update rust toolchain version. The latest nightly doesn't support json targets without passing some argument to rustc. So don't use the latest. concat_ident! is removed in favor of concat!, and we are actually already using that. So remove the unstable feature declaration in src/lib.rs as well. Signed-off-by: greatbridf <greatbridf@icloud.com>
Change the type of target width items to numbers according to the specifications. Signed-off-by: greatbridf <greatbridf@icloud.com> (cherry picked from commit a5f6349) Signed-off-by: greatbridf <greatbridf@icloud.com>
External interrupts were unimplemented on loongarch64: the trap context
mapped HWI0..HWI5 to a `todo!()`, so no driver could ever be woken by
hardware. This made the serial port unusable for input -- the read
worker armed the UART's receive interrupt and then waited forever.
On the QEMU virt machine the path from a device to the CPU is
device -> PCH-PIC line -> EIOINTC vector -> CPU HWI line
The PCH-PIC is an ordinary MMIO block at 0x1000_0000; the EIOINTC lives
in the IOCSR address space and is accessed with iocsrrd/iocsrwr. Add a
small interrupt module that programs both: every EIOINTC vector is
routed to HWI1 (matching the interrupt controller's `interrupts = <3>`
device tree entry) and to CPU 0, and PCH-PIC lines are additionally
routed to the vector of the same number through HTMSI_VEC.
Lines stay masked until a handler exists. register_irq_handler() now
unmasks the corresponding PCH-PIC line through the HAL on loongarch64,
so an interrupt cannot be delivered before the kernel can service it,
and enabling a line clears any stale EIOINTC pending bit first.
dispatch_irq() acknowledges the pending bit in COREISR before invoking
the handler, so a peripheral that keeps its line asserted simply
re-raises the interrupt until it is drained. Only vectors 0..32 are
dispatched; the remaining vectors belong to the PCI MSI controller,
which is not supported yet, so they are acknowledged and dropped.
With this in place UART0 (PCH-PIC line 2) delivers receive interrupts
and serial input works under QEMU: typed commands are echoed and
executed, and the system stays idle at the prompt instead of spinning
on a level-triggered interrupt.
Assisted-by: deepseek-flash
Signed-off-by: greatbridf <greatbridf@icloud.com>
Rearrange code to pass args instead of by envs, no functionality changes. Signed-off-by: greatbridf <greatbridf@icloud.com>
Signed-off-by: greatbridf <greatbridf@icloud.com>
- printf -v is non-standard, use var=$(printf) instead. - expect is not installed in the runner. install it manually Signed-off-by: greatbridf <greatbridf@icloud.com>
Add PLATFORM in Makefile to indicate the actual system setup. Put build image script of each platforms to dedicated folders. This makes creating test setups easier. Signed-off-by: greatbridf <greatbridf@icloud.com>
No functional change. Signed-off-by: greatbridf <greatbridf@icloud.com>
Instead, add an example vscode settings file for IDE use. Signed-off-by: greatbridf <greatbridf@icloud.com>
Signed-off-by: greatbridf <greatbridf@icloud.com>
Add a project-scoped Pi skill that guides kernel issue reproduction, root-cause analysis, fix verification, and PR submission. The skill requires GH_TOKEN before doing any work and uses phase-gated subagent handoffs. Split the detailed protocol into phase-specific reference documents so only the relevant guidance is loaded for each phase. Signed-off-by: greatbridf <greatbridf@icloud.com>
- Add script/backtrace to translate backtraces. - Add a cut sign in the kernel panic routine to indicate the start of stack backtrace. Signed-off-by: greatbridf <greatbridf@icloud.com>
Inode and superblock rework: Remove old Inode trait as it used to undertake too much responsibility. The new method use three new traits: InodeOps is used to acquire generic inode attributes. InodeFileOps and InodeDirOps handle file and directory requests respectively. All the three have async fn trait methods and don't need to be ?Sized. Then, we implement Inode, InodeFile and InodeDir for the implementors of the three "Ops" traits, erasing their actual type and provide generic dyn interface by wrapping the futures in boxes. We should provide an io worker? or some io context with an allocator for futures to reduce the overhead of io requests, or come up with some better ideas? For inode usage, we introduce InodeRef and InodeUse. InodeRef is a simple wrapper for Weak<impl Inode> and InodeUse for Arc<impl Inode>. This helps us use them better as we can't define impls for Arc<dyn Inode>'s as they are foreign types. We also provide some more helper methods for them. After the change, we don't impose ANY structural restriction except for the spinlock wrapped InodeInfo. The InodeInfo struct design might need rethinking but the current implementation seems to be fine aside from unnecessary locks when we try to access some of its fields but this shouldn't be a VERY big problem and very urgent... Similar changes are also made to superblock traits and types. But for the superblock objects, we use a SuperBlockComplex struct to store common fields such as whether the superblock is read only, their device id and so on. Also the structs have a superblock rwsem inside. But we haven't decided how to use that (such as whether we should acquire the lock and pass it to the inode methods) and even whether they should exist and be there. This will need further thinking so we put this off for now... Filesystem rework: Rework tmpfs, fatfs and procfs with the new technology mentioned above, leaving the old ext4 unchanged. The current implementation of ext4 uses some "random" library from the "camp". Its code hasn't been fully reviewed for time reasons but seems to be rather "problematic"... We might rewrite the whole module later and the page cache subsystem requires fully reworking as well. So we put this work off as well. Block device and other parts rework: Wraps PageCacheBackend, MountCreator and BlockRequestQueue with async_trait to provide dyn compatible async functions. Dentry walking functions are also put to the heap since they are recursive functions... This has similar problems to the inode traits, ugly solution. Further optimization is required. Signed-off-by: greatbridf <greatbridf@icloud.com>
The old path walking algorithm requires recursion, which is not supported in async rust. So we boxed them all as a temporary solution in previous commits. This would introduce mass overhead even for fast path walks just because we might sleep in `readlink()` and `lookup()` calls. The new proposed method is to break the walk into several phases similar to that in Linux: RCU walk and REF walk. The RCU walk will hold the RCU lock and never blocks so the function itself can be non-async. If we hit non-present dentries, we will fallback to REF walk. In REF walks, we clone the Arcs and consult to the VFS layer for an accurate answer. Note that in both the two methods mentioned above, symlinks are not handled and will be returned directly with all path components left untouched. We have a dedicated async function to follow the symlinks by recursive calling the walk function. This can be slow and won't be called frequently. So we wrapped the function with `Box::pin()` to break the recursion chain. After the symlink resolution is done, we return to the original position and continue the walk. We found that the association of an inode to a dentry is one way. So the `data` RCUPointer field is actually unnecessary and we can use the atomic dentry type to sync readers with the writer. This way we can eliminate `DentryData` allocations and improve performance. We also introduced a new RCU read lock syntax. In the RCU walk mentioned above, we need to store dentry references protected by some RCU read lock. With the old syntax, we can't express the lifetime associtated by the common RCU read lock. The new syntax provides a `rcu_read_lock()` method to acquire the RCU read lock. The lock returned has a associated lifetime so we can use it in the RCU session. Signed-off-by: greatbridf <greatbridf@icloud.com>
Signed-off-by: greatbridf <greatbridf@icloud.com>
If we have invalid format, we should print a None instead of panicking. Signed-off-by: greatbridf <greatbridf@icloud.com>
Signed-off-by: greatbridf <greatbridf@icloud.com>
Introduce the new page locking mechanism to ensure exclusiveness of pages when we access them. The underlying locks are not implemented yet for now because we will change the paging structs in the following few patches. Introduce a new `PageExcl` struct representing a page that conforms with Rust's ownership rules. The page owned exclusively can be accessed without taking page locks. Remove the `MemoryBlock` structs as they are not easy to use and represent barely no semantic meanings. Signed-off-by: greatbridf <greatbridf@icloud.com>
Separate old UserTLS into UserTLS and UserTLSDescriptor. UserTLS is for threads to hold infomation about its storage. Descriptors are used in clone syscalls. Signed-off-by: greatbridf <greatbridf@icloud.com>
Signed-off-by: greatbridf <greatbridf@icloud.com>
Introduce `Zone`s: a Zone is a region of physical memory that all share the same NUMA node. The Zone will hold all RawPage structs. Buddy allocator will now store a reference to the zone that hold all its pages. Thus, we make the buddy allocator independent of underlying physical page frame management framework. Remove unnecessary page flags and structs. Signed-off-by: greatbridf <greatbridf@icloud.com>
- Bump rust compiler version to nightly-2026-01-09. - Inode rework: add a generic Inode struct. - Add a macro to help function tweaks. - PageCache rework: reduce complexity and try to decouple. - Adapt fat32, tmpfs to the new page cache system. - Change the way we process mapped pages and load ELF executables. - Refine handling flags in `MMArea::handle_mmap`. Signed-off-by: greatbridf <greatbridf@icloud.com>
- Remove struct `Page` and add `Folio`s to represent adjacent pages. - Introduce `Zone`s similar to that in Linux. Looking forward to removing all occurrence of `ArchPhysAccess` and so on. - Adapt existing code to new `Folio` interface in a dirty and rough way. Signed-off-by: greatbridf <greatbridf@icloud.com>
No functional changes
- Add `{extern_,}symbol_addr` macro to retrieve symbol address.
- Remove manual impl Send and Sync for RawPage
- Make elided '_ lifetimes in return types explicit
- Suppress unused warnings by allowing them
- Remove really unused functions
- Refactor `println_trace` macro to suppress unused variable warnings
Signed-off-by: greatbridf <greatbridf@icloud.com>
Make print message prettier. Also panic when we have an error. Signed-off-by: greatbridf <greatbridf@icloud.com>
- thd.exit() will set thd.dead and send it to the reaper - delay the release of process mm until we reap it - extract futex logic out of exit and exec routine Signed-off-by: greatbridf <greatbridf@icloud.com>
- Use intrusive lists to store and organize the process hierarchy. - Remove `FileArray::open_console()`. Do it in the init script instead. - Fix open logic: acquire controlling terminals only if O_NOCTTY is not set. Put this into TerminalFile::open(). - Send SIGHUP and then SIGCONT to foreground pgroup procs when the controlling terminal is dropped. - Set the controlling terminal of sessions in Terminal. - Limit max line width to 80. Format some codes. Signed-off-by: greatbridf <greatbridf@icloud.com>
Reformat the files with new the format style to make the real changes clearer. Signed-off-by: greatbridf <greatbridf@icloud.com>
We already have `FDT.present_ram().free_ram()`. Remove the impl in `ArchMemory` to avoid confusion. Signed-off-by: greatbridf <greatbridf@icloud.com>
- Add `extern_symbol_value` to retrieve far relative symbol values. - Get `BSS_LENGTH` and `__kernel_end` using `extern_symbol_addr`. - Get `_ap_start` using `extern_symbol_value`. Signed-off-by: greatbridf <greatbridf@icloud.com>
In ea21223 ("vfs, rcu: rework path walking with new rcu syntax"), we introduced the `RCUReadGuardNew` lock complex, which encapsulates RCU guard and the preempt guard together. We should reuse the existing interface to do RCU read locks. Signed-off-by: greatbridf <greatbridf@icloud.com>
No functional change. Signed-off-by: greatbridf <greatbridf@icloud.com>
Instead of acquiring the spinlock at the begin and hold it throughout the `ThreadBuilder::clone_from()`, we lock and use it only when we need to access `inner.name`. This can reduce contention. Signed-off-by: greatbridf <greatbridf@icloud.com>
If TLS is not given in clone syscall arguments, the userspace programs may read undefined value on some platforms (for example, x86, because the TLS is not stored in TrapContexts). And this causes NULL pointer derefs when the userspace program uses the TLS. When we are building a thread, use the parent's TLS if not given from `CloneArgs` to address this problem. Fixes: 5bcdb80 ("feat: implement clone") Signed-off-by: greatbridf <greatbridf@icloud.com>
Add a macro to define late init functions. Function addresses are collected into an array and called after kernel initialization. Async functions can also be done in the future: just return a boxed Future in the helper function. Signed-off-by: greatbridf <greatbridf@icloud.com>
Add another array storing async function stubs that wrap the async function with `Box::pin()`. Call these functions after system initialization. Signed-off-by: greatbridf <greatbridf@icloud.com>
No functional change. Signed-off-by: greatbridf <greatbridf@icloud.com>
No functional change. Signed-off-by: greatbridf <greatbridf@icloud.com>
Now that we have late init, we can use this instead of contaminating the global namespace. Signed-off-by: greatbridf <greatbridf@icloud.com>
Signed-off-by: greatbridf <greatbridf@icloud.com>
This was used in `dump_mount()` and is now unused after rewrite of `dump_mount()` in commit 181c189 ("vfs: move /proc/mounts population to mount.rs")"). Remove it now. Signed-off-by: greatbridf <greatbridf@icloud.com>
These macros were kept for compatibility and is unused now. Remove them. Signed-off-by: greatbridf <greatbridf@icloud.com>
Signed-off-by: greatbridf <greatbridf@icloud.com>
We hope to completely remove prelude module in the future. Do some of the cleanup work now. Signed-off-by: greatbridf <greatbridf@icloud.com>
Signed-off-by: greatbridf <greatbridf@icloud.com>
`Buffer`s return `core::fmt::Result` when it overflows. The bothersome error handling code can be encapsulated in a macro. Besides, we don't have to import `core::fmt::Write` trait manually now. Signed-off-by: greatbridf <greatbridf@icloud.com>
Make `make_slab_page()` private since it's not used outside. Signed-off-by: greatbridf <greatbridf@icloud.com>
SlabList have exclusiveness guarantee and we don't shrink slab page list for now. So we just: - Increase total folio and object count when new slabs are allocated. - Increase active folio count when we grab a new empty slab. - Increase active object count when allocation succeed. - Decrease active folio count when we push slabs back to empty list. - Decrease active object count on deallocation. and all should be fine. Signed-off-by: greatbridf <greatbridf@icloud.com>
- Add `SlabListStat` for debug use. - Add a function and copy the info from the list in the function. - Add /proc/slabinfo as a userspace interface, the format of which taken from Linux. Signed-off-by: greatbridf <greatbridf@icloud.com>
With slight style change, no functional change. Signed-off-by: greatbridf <greatbridf@icloud.com>
Add a counter in each of `FreeArea`s. Increase the counter on page allocation and decrease on deallocation. Signed-off-by: greatbridf <greatbridf@icloud.com>
No functional change. Signed-off-by: greatbridf <greatbridf@icloud.com>
I notice a memory leak when the task is finished and we store a None to CURRENT_TASK without retrieving and release the arc refcount. Fix this by introducing `Task::_swap_current()` to swap and return the previous Option<Arc<Task>> to actually drop it. Replace other operations to CURRENT_TASK with this function as well. Fixes: e23c9eb ("runtime: new task sleep-wakeup method and some adaption")") Signed-off-by: greatbridf <greatbridf@icloud.com>
No functional change. Signed-off-by: greatbridf <greatbridf@icloud.com>
When we remove processes in wait syscalls, we forget to remove the thread from the process's threads list, which will introduce a memory leak since the Arc<Thread> will never get released. Fix the issue by adding a `remove_thread` call to release it. Fixes: 6695889 ("rewrite: move Process, ProcessGroup, Session out of thread.rs")") Signed-off-by: greatbridf <greatbridf@icloud.com>
To use `buffer.get_writer()`, we need to import Buffer trait. But for users using buf_writeln macros, this can be confusing. add the use statement inside the macro can solve this. Signed-off-by: greatbridf <greatbridf@icloud.com>
In future patch of supporting buddyinfo procfs file, we need to print one line in many write calls. Support this by copying buf_writeln definition and replace `writeln!` with `write!`. Signed-off-by: greatbridf <greatbridf@icloud.com>
Similar to what we've done to the slab system, we count allocated and total folio count and add a procfs file to show stat. Signed-off-by: greatbridf <greatbridf@icloud.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
Bulk of patches related to everything that I have no spare time to catagorize.
Just merge them all in one go. AI generated messages below.
Summary
This branch is a broad rework of Eonix's core subsystems: the VFS and page
cache are now async and driven by a new inode/superblock model,
physical memory is managed through
Zone/Folioabstractions with reworkedbuddy and slab allocators, process/thread bookkeeping was rebuilt around
intrusive lists and an exited-thread reaper, and all three architectures now
share a captured-trap model (including a rewritten x86_64 trap path). It also
adds a kernel config/late-init mechanism, bumps the toolchain to
nightly-2026-01-09, and rebuilds the test and image tooling.112 commits, 220 files changed, +13,046 / -8,698.
Motivation
Inodetrait and recursive path walking,which cannot be expressed in
asyncRust and forced heap-boxed futures evenon the hot path. The page cache was tightly coupled to filesystem code.
Pagetype with no notion ofzones or multi-page allocations, and slab/buddy allocators had no
debug/accounting facilities.
TaskContextswitch code and trap entryconventions. x86_64 in particular needed a dedicated scheme around its
limited register set, and had no shutdown support.
image, CPU counting fixes, and a working vDSO/sigreturn path.
What changed
VFS: async inode/superblock model and new page cache
Inodetrait withInodeOps/InodeFileOps/InodeDirOps, plusInode,InodeFile,InodeDiradapter types that eraseconcrete filesystem types and box futures at the dyn boundary
(
675c9f11,ef42bc6c).InodeUse(Arc-backed inode handle) and the weak/strongsuperblock handles
SbRef/SbUse, plus a genericInodestruct withatomic mode/nlink and a locked
InodeInfo.SuperBlockComplex, and moved filesystemregistration and
/proc/mountspopulation intomount.rsusing late init(
5fe394c5).fat32, tmpfs and procfs were ported to it, and mapped-page / ELF loading now
goes through the new cache (
ef42bc6c).CharDevice::read()async and adapted the terminal/VFS plumbing(
6d05a0f8).a REF walk on missing dentries, with symlink following delegated to a
dedicated boxed future (
572333f9, newvfs/dentry/walk.rs). DroppedDentryDataallocations since inode-to-dentry association is one-way; thenew
arcrefcrate (ArcRef) was added in the same commit to support this.rcu_read_lock()syntax whose guard ties dentry references tothe RCU session (
10eb2c4b).Memory management: zones, folios, buddy and slab
Zone(physical memory sharing a NUMA node) owning allRawPages, making the buddy allocator independent of the page-frameframework (
7792a5e8).Folioabstraction for one or more adjacent pages, replacingPage/MemoryBlock, withLockedFolioguards andFolioOwnedforexclusive ownership (
96745750).03811435) and added accounting/debugreporting for slab caches, allocated objects and buddy pages
(
2e2f57b3,5ae4a087,cac34352,1228689b).MMArea/MMList: per-area locks, anAreaListlock, a zero-sizedMemListLock, atomicMemAreaflags, andMemArea::grow/clone/unmap/protectimplemented viaisolate()with intrusiveMemArealists(
37b8f249…b401a073).brkhandling intomm_list/brk.rs(6e42a537).Tasks, processes and runtime
terminal/session handling, and send
SIGHUP+SIGCONTto the foregroundprocess group when the controlling terminal is dropped (
96803aab).mmrelease until reaping; futexlogic was extracted out of
exit/exec(fed6f4ac).clone call does not supply one, and reduced the time the thread inner
spinlock is held (
be57074c,783a2ffa).UserTLSintoUserTLS(per-thread storage) andUserTLSDescriptor(used by clone), with per-arch implementations(
df7c7300,6204f199).runtime release finished
Tasks instead of leaking them(
f3525ae0,65531a95).functions (
8a0dfcdb).HAL and architecture support
TaskContextswitch on x86_64 and riscv64 with aunified
TrapContext::newand a captured-trap model; x86_64 now savesscratch registers in per-CPU storage and performs context switches with
handwritten entry/return code (
593b871b,a700ab8d,f95c05fe,63e8e992,a51f68d7). loongarch64 still uses its ownTaskContextswitch for now.
c36e6e8b).1acdf5cd, later moved intothe
powermodule in7ae60df2), support for kernel images up to 14 MiBusing huge mappings (
cd95937d), exported symbols required by the vDSO, andsupport for the new folio abstraction.
CPU_COUNTnever being incremented insetup_cpu(
594b358f, fixesdd32b939).free-memory pool, fixing sporadic freezes on process creation/exit
(
89f277cf, closes Random kernel freezing on process creation / exiting #54).5071f129).build.rs(
arch_has_stacktrace,arch_has_shutdown) and consolidated panic handlinginto
src/panic.rs(194624a9).Kernel init and tooling
late_initmechanism:#[define_late_init]functions are collectedinto a linker array and run after normal init, with a second array for async
variants (
b7ddbea3,8a0dfcdb).nightly-2026-01-09and updated the x86_64 targetspec (
3edf55f5,e49012a1,9d270c72).script/build-img.shinto per-platformscript/build-img/modulesand added a
PLATFORMMakefile variable (dc8dc98b,2202f183).script/test.sh(expect-driven QEMU smoke test:ls, root-listingcheck, echo a magic string,
poweroff) and the GitHub Actions workflow(
2e20ee7f,731798d0).(
e0cb58da).script/backtracefor symbolizing kernel stack traces, plus thefix-and-pragent skill, a.vscodesettings example, andbuf_write/buf_writelnhelpers (96c15ee7,1149777f).(
f0184876,710d9a00,88ec9623).Breaking changes and currently disabled features
src/fs/ext4.rshas notbeen ported to the new async VFS and remains commented out;
src/fs/shm.rsis removed, so
shmget/shmat/shmdt/shmctlhandlers are gone (thesyscall numbers remain defined). Both are tracked as follow-up work.
Inodetrait is split into*Opstraits; filesystem implementationsmust be updated.
TaskContextis removed on x86_64 and riscv64; out-of-tree arch code forthose targets must move to
TrapContext. loongarch64 keepsTaskContextuntil it is migrated to the same model.
MMArea/MMListinternals changed substantially (per-area locks,intrusive lists).
Testing
riscv64gc-unknown-none-elf(virt) andx86_64-unknown-none(acpi_compat) with the pinned nightly toolchain.script/test.shsmoke test (busyboxls, check thatprocappears in theroot listing, echo round trip,
poweroff)..github/workflows/test-build.yaml) builds release images for the testmatrix and runs the same smoke test; run logs are uploaded as artifacts.
Closes #54(riscv64 random freeze during process creation/exit).Review guide
The branch is large but mostly separable:
675c9f11,572333f9,ef42bc6c— VFS/inode/page-cache rework.96745750,7792a5e8,03811435,37b8f249…b401a073— memory rework.96803aab,fed6f4ac,df7c7300— process/thread/TLS rework.593b871b,a700ab8d,f95c05fe— trap handling across architectures.b7ddbea3,8a0dfcdb,194624a9— late init, config and panic.style:reformats, build scripts and CI.Follow-ups
unmapcurrently leaves apage-recycling TODO) and settle the superblock locking model.
allocator) and in symlink resolution.
(
arch: [riscv64, loongarch64]), since the newerincludeentry only addsx86_64 and leaves the loongarch64 job with an empty target/platform.
TrapContextmodel.