Re: [OSDev] How to switch to long mode in x86 CPUs?

[Note: Followup-To: set to alt.os.development]

In article <871pvje5yq.fsf@onesoftnet.eu.org>,
Ar Rakin  <rakinar2@onesoftnet.eu.org> wrote:

Hello there,
>
I am trying to develop my own, simple operating system to learn more
about how kernels work and low level stuff like that.  However, I am
stuck at setting up paging while switching long mode (64-bit protected
mode) in x86 processors.

As has been mentioned, comp.lang.c is not the appropriate place
to ask this.  I have set the `Followup-To:` header to
alt.os.development, and am cross-posting this post to that
newsgroup.

The assembly code I currently have:
>
#define PG_START 0x000000000

Just to be clear, this means that you have decide to make your
_virtual_ address space starts at absolute address 0?  What
address do you link your kernel at?

#define MSR_EFER 0xc0000080
>
.section .bss, "aw", @nobits
.align 4096
pml4_tbl:
 .skip 4096
pdpt_tbl:
 .skip 4096

This is fine, but note that, instead of using code to fill in
your page tables, you could simply define them here with the
expected entries, as they are very simple.

.text
.globl _mboot_start
_mboot_start:
 /* GRUB executes this code in 32-bit protected mode. */
>
 /* Write (pdpt_tbl | 0x3) to the first 8 bytes of pml4_tbl */
 movl $pdpt_tbl, %eax
 orl $0x3, %eax
 movl $pml4_tbl, %edi
 movl %eax, (%edi)

Note that this sequence implicitly assumes that you are starting
with an identity mapping between between the physical and
virtual address spaces.  In particular, when you
`movl $pdpt_tbl, %eax` you are copying whatever address the
linker assigns to `$pdpt_tbl` into %eax (the low 32-bits of it
anyway, though the assembler would probably sqwuak at you if
didn't fit into a 32 bit immediate).  Page table entries must
refer to physical addresses, so if you've arranged for the
linker to use some base address other than 0 for your kernel,
you've got to take care to account for an offset here.

 xorl %eax, %eax
 movl %eax, 4(%edi)

Note that, as you're doing this in assembly, the upper bits in
the table are already filled with zeros, so there's no need for
the `xorl %eax, %eax; movl %eax 4(%edi)` sequence.

 movl $pdpt_tbl, %edi
 movl $PG_START, %eax
 /* 0x83 = 0b10000011; flags: present, writable, upervisor-only,
    1GB huge page */
 movl $0x83, (%edi)
 movl %eax, 4(%edi)

This looks correct.  Your page tables will now map a single
gigabyte of address space starting at (virtual) address zero to
physical address 0, and nothing else.  To be clear, is that what
you want?  When coming out of protected mode, I generally try to
map the whole 32-bit address space; that is, all 4 GiB.

 /* Enable Physical Address Extension (PAE) */
 movl %cr4, %eax
 btsl $5, %eax
 movl %eax, %cr4
>
 /* Load the address of the PML4 table into %cr3 */
 movl $pml4_tbl, %edi
 movl %edi, %cr3

Note that the same caveat about physical addresses of the PML4
apply here as applied to the PDPT above.

 /* Enable long mode */
 movl $MSR_EFER, %ecx
 rdmsr
 btsl $8, %eax
 wrmsr
>
 /* Enable paging */
 movl %cr0, %eax
 btsl $31, %eax
 movl %eax, %cr0

So immediately after executing this instruction, the processor
will be executing with paging enabled.  This means that the very
next instruction (the ljmp below here) _must_ be mapped at the
address in %rip.  If not, you will fault.

Your post suggests that you fault on the ljmp, but this may not
be why.  Any easy test would be to add a `jmp .` here and see if
that faults in the QEMU monitor.

If you do not fault here, then your page tables are ok (at least
so far), and your problem lies elsewhere.  See below.

 /* Jump to 64-bit code */
 ljmpl $0x08, $long_mode_entry

Have you set up a GDT with an entry for a 64-bit code segment
by this point?  It doesn't look like it.  My guess is that that
is the source of your fault; note that the multiboot1 spec says
that you must set up a GDT and should not rely on the one that
it set up to get you into 32-bit protected mode.  Certainly
there is no guarantee that there's a 64-bit code segment at
offset 0x8 in whatever table it set up.

My guess is that this is the source of your problem.

.loop:
 hlt
 jmp .loop

I would delete this loop; you can't ever really hit it: either
the long jump will succeed and skip over it, or it will fault.

long_mode_entry:
 .code64
 xorw %ax, %ax
 movw %ax, %ds
 movw %ax, %es
 movw %ax, %fs
 movw %ax, %gs
 movw %ax, %ss
 
 callq kmain

You should probably give yourself a stack before calling C.
What's in `%rsp` here?  My guess is that this would fault if you
got here: the `callq` will push the address of the next
instruction onto the stack, but since you haven't set one up,
%rsp is whatever it is (either it's reset value, 0, or something
random set up by multiboot).  Suppose it's 0; then the call will
attempt to push %rip to -8; that's fine (the processor will
happily wrap around to 0xfffffffffffffff8) but you definitely
don't have anything mapped there, so you'll get a fault.

In 32-bit mode, this will wrap around to 0xfffffff8, which is
in the active stack segment; there may be RAM there, which is
why it doesn't fault in protected mode.

Since you haven't set up an IDT yet, any fault this early will
be a triple fault.

Or perhaps multiboot left you a stack by chance.  The multiboot1
specification explicitly says that the OS should set up it's own
stack, though, so I wouldn't rely on that.

Try adding a page for more for stack space in your BSS, and set
%rsp to point to the top of that region, e.g., back in bss:

stack:
.skip 4096

...

movabsq $(stack + 4096), %rsp
callq kmain

 callq kabort

I take it you never get here.  :-)

I am not sure what is wrong, but when I run my kernel in
qemu-system-x86_64, it causes a triple fault when trying to jump to the
long mode code.  After a lot of debugging, I am sure that the issue is
with paging, because removing the ljmpl and paging instructions do not
cause any further errors and the kernel runs fine in 32-bit mode.

My guess is that, assuming you're correctly referring to the
physical address of your page table structures as I mentioned
above,  paging is fine, but rather, it's segmentation that's
causing the problem.

If anyone knows what is wrong with this code, please let me know.  Any
help will be appreciated!

At a minimum, set up a GDT for yourself and give yourself a
proper stack before calling into C code; a lot of context, in
particular your linker configuration, is missing so it's hard to
say whether those are the only problems.

You may want to consider using named constants in place of the
manifest values you've got now when setting bits and so forth.

- Dan C.