Discussion:
running fs.py with X86KvmCPU failed
(too old to reply)
Da Zhang
2018-02-08 04:48:13 UTC
Permalink
I am trying to run fs.py with kvm support which might help speedup
our simulation in full system mode. I find the cpu type X86KvmCPU which is
a "kvm-based hardware virtualized cpu". But running fs.py failed with the
error information:

panic: KVM: Failed to enter virtualized mode (hw reason: 0x80000021)

Memory Usage: 2416600 KBytes

Program aborted at tick 53418967500

--- BEGIN LIBC BACKTRACE ---

build/X86/gem5.fast(_Z15print_backtracev+0x1f)[0xaee60f]

build/X86/gem5.fast(_Z12abortHandleri+0x34)[0xaee6f4]

/lib64/libpthread.so.0(+0xf5e0)[0x7f5b9ac685e0]

/lib64/libc.so.6(gsignal+0x37)[0x7f5b9901c1f7]

/lib64/libc.so.6(abort+0x148)[0x7f5b9901d8e8]

build/X86/gem5.fast[0x6627df]

build/X86/gem5.fast[0x95d518]

build/X86/gem5.fast(_ZN10BaseKvmCPU13handleKvmExitEv+0x249)[0xc10859]

build/X86/gem5.fast[0xbade3c]

build/X86/gem5.fast(_ZN10EventQueue10serviceOneEv+0x91)[0xad4fc1]

build/X86/gem5.fast(_Z9doSimLoopP10EventQueue+0xa0)[0xb5b110]

build/X86/gem5.fast(_Z8simulatem+0x1f3)[0xb5b563]

build/X86/gem5.fast[0x93867d]

build/X86/gem5.fast[0x939c65]

/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x730a)[0x7f5b9a56b0ca]

/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]

/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x663c)[0x7f5b9a56a3fc]

/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x67bd)[0x7f5b9a56a57d]

/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x67bd)[0x7f5b9a56a57d]

/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]

/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f5b9a56d002]

/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5513)[0x7f5b9a5692d3]

/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]

/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x663c)[0x7f5b9a56a3fc]

/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]

/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f5b9a56d002]

/lib64/libpython2.7.so.1.0(+0x10043f)[0x7f5b9a58643f]

/lib64/libpython2.7.so.1.0(PyRun_StringFlags+0x65)[0x7f5b9a5872a5]

build/X86/gem5.fast(_Z6m5MainiPPc+0x5f)[0xad327f]

build/X86/gem5.fast(main+0x33)[0x5ddf23]

/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f5b99008c05]

build/X86/gem5.fast[0x5df8fc]

--- END LIBC BACKTRACE ---

Aborted (core dumped)

the command is as simple as:

build/X86/gem5.fast -d ~/tmp/output1/ configs/example/fs.py --mem-size=2GB
--disk-image=linux-x86.img --cpu-type=X86KvmCPU

Any idea? thanks in advance.

best,
Da
Jason Lowe-Power
2018-02-08 17:01:02 UTC
Permalink
These patches "fix" the problem. However, they may not apply cleanly to
HEAD and they definitely are not cleanly implemented.

https://gem5-review.googlesource.com/c/public/gem5/+/7362
https://gem5-review.googlesource.com/c/public/gem5/+/7361

Cheers,
Jason
Post by Da Zhang
I am trying to run fs.py with kvm support which might help speedup
our simulation in full system mode. I find the cpu type X86KvmCPU which is
a "kvm-based hardware virtualized cpu". But running fs.py failed with the
panic: KVM: Failed to enter virtualized mode (hw reason: 0x80000021)
Memory Usage: 2416600 KBytes
Program aborted at tick 53418967500
--- BEGIN LIBC BACKTRACE ---
build/X86/gem5.fast(_Z15print_backtracev+0x1f)[0xaee60f]
build/X86/gem5.fast(_Z12abortHandleri+0x34)[0xaee6f4]
/lib64/libpthread.so.0(+0xf5e0)[0x7f5b9ac685e0]
/lib64/libc.so.6(gsignal+0x37)[0x7f5b9901c1f7]
/lib64/libc.so.6(abort+0x148)[0x7f5b9901d8e8]
build/X86/gem5.fast[0x6627df]
build/X86/gem5.fast[0x95d518]
build/X86/gem5.fast(_ZN10BaseKvmCPU13handleKvmExitEv+0x249)[0xc10859]
build/X86/gem5.fast[0xbade3c]
build/X86/gem5.fast(_ZN10EventQueue10serviceOneEv+0x91)[0xad4fc1]
build/X86/gem5.fast(_Z9doSimLoopP10EventQueue+0xa0)[0xb5b110]
build/X86/gem5.fast(_Z8simulatem+0x1f3)[0xb5b563]
build/X86/gem5.fast[0x93867d]
build/X86/gem5.fast[0x939c65]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x730a)[0x7f5b9a56b0ca]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x663c)[0x7f5b9a56a3fc]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x67bd)[0x7f5b9a56a57d]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x67bd)[0x7f5b9a56a57d]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f5b9a56d002]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5513)[0x7f5b9a5692d3]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x663c)[0x7f5b9a56a3fc]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f5b9a56d002]
/lib64/libpython2.7.so.1.0(+0x10043f)[0x7f5b9a58643f]
/lib64/libpython2.7.so.1.0(PyRun_StringFlags+0x65)[0x7f5b9a5872a5]
build/X86/gem5.fast(_Z6m5MainiPPc+0x5f)[0xad327f]
build/X86/gem5.fast(main+0x33)[0x5ddf23]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f5b99008c05]
build/X86/gem5.fast[0x5df8fc]
--- END LIBC BACKTRACE ---
Aborted (core dumped)
build/X86/gem5.fast -d ~/tmp/output1/ configs/example/fs.py --mem-size=2GB
--disk-image=linux-x86.img --cpu-type=X86KvmCPU
Any idea? thanks in advance.
best,
Da
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
Da Zhang
2018-02-08 20:50:36 UTC
Permalink
Hi Jason

The package works (I used the second one)! And it also works with the
package you provided (
https://gem5-review.googlesource.com/c/public/gem5/+/7301) in my another
email to fix the keyboard and mouse issue for running later linux kernel
and ubuntu. (However, there are some conflicts in this package, and it is a
little tricky to merge them.)

Now, I can run gem5 for linux kernel v4.8.13 and ubuntu 16.04.1 with
kvm support. And the speedup is so amazing. It used to take me 20 ~ 30
minutes to boot up the system without the kvm cpu. Now, it takes only
several seconds!!!

Thanks so much!

best,
Da
Post by Jason Lowe-Power
These patches "fix" the problem. However, they may not apply cleanly to
HEAD and they definitely are not cleanly implemented.
https://gem5-review.googlesource.com/c/public/gem5/+/7362
https://gem5-review.googlesource.com/c/public/gem5/+/7361
Cheers,
Jason
Post by Da Zhang
I am trying to run fs.py with kvm support which might help speedup
our simulation in full system mode. I find the cpu type X86KvmCPU which is
a "kvm-based hardware virtualized cpu". But running fs.py failed with the
panic: KVM: Failed to enter virtualized mode (hw reason: 0x80000021)
Memory Usage: 2416600 KBytes
Program aborted at tick 53418967500
--- BEGIN LIBC BACKTRACE ---
build/X86/gem5.fast(_Z15print_backtracev+0x1f)[0xaee60f]
build/X86/gem5.fast(_Z12abortHandleri+0x34)[0xaee6f4]
/lib64/libpthread.so.0(+0xf5e0)[0x7f5b9ac685e0]
/lib64/libc.so.6(gsignal+0x37)[0x7f5b9901c1f7]
/lib64/libc.so.6(abort+0x148)[0x7f5b9901d8e8]
build/X86/gem5.fast[0x6627df]
build/X86/gem5.fast[0x95d518]
build/X86/gem5.fast(_ZN10BaseKvmCPU13handleKvmExitEv+0x249)[0xc10859]
build/X86/gem5.fast[0xbade3c]
build/X86/gem5.fast(_ZN10EventQueue10serviceOneEv+0x91)[0xad4fc1]
build/X86/gem5.fast(_Z9doSimLoopP10EventQueue+0xa0)[0xb5b110]
build/X86/gem5.fast(_Z8simulatem+0x1f3)[0xb5b563]
build/X86/gem5.fast[0x93867d]
build/X86/gem5.fast[0x939c65]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x730a)[0x7f5b9a56b0ca]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x663c)[0x7f5b9a56a3fc]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x67bd)[0x7f5b9a56a57d]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x67bd)[0x7f5b9a56a57d]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f5b9a56d002]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5513)[0x7f5b9a5692d3]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x663c)[0x7f5b9a56a3fc]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f5b9a56d002]
/lib64/libpython2.7.so.1.0(+0x10043f)[0x7f5b9a58643f]
/lib64/libpython2.7.so.1.0(PyRun_StringFlags+0x65)[0x7f5b9a5872a5]
build/X86/gem5.fast(_Z6m5MainiPPc+0x5f)[0xad327f]
build/X86/gem5.fast(main+0x33)[0x5ddf23]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f5b99008c05]
build/X86/gem5.fast[0x5df8fc]
--- END LIBC BACKTRACE ---
Aborted (core dumped)
build/X86/gem5.fast -d ~/tmp/output1/ configs/example/fs.py
--mem-size=2GB --disk-image=linux-x86.img --cpu-type=X86KvmCPU
Any idea? thanks in advance.
best,
Da
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
Da Zhang
2018-02-13 15:51:16 UTC
Permalink
Hey Jason,

The package works.
However, I encountered performance issues with increased CPU number. The
performance was very great with up to 4 CPUs ("-n", I am assuming "number
of CPUs" equal to "number of cores" for testing multithreading workload
later). However, the system fails to boot (or maybe it was just too slow)
when I scale it >= 5 CPUs. Any ideas or suggestions?

I am working on a node with 2 x Intel(R) Xeon(R) CPU E5-2470 v2 @ 2.40GHz,
10 physic cores each (and 40 logical cores in total). I only used the fs.py
script with --cpu-type=X86KvmCPU, --mem-size=8GB and -n 4. Moreover, the
ancient Linux kernel and image from gem5 website have no performance
problem with increased CPUs.

best,
Da Zhang
Post by Da Zhang
Hi Jason
The package works (I used the second one)! And it also works with the
package you provided (https://gem5-review.googlesource.com/c/public/
gem5/+/7301) in my another email to fix the keyboard and mouse issue for
running later linux kernel and ubuntu. (However, there are some conflicts
in this package, and it is a little tricky to merge them.)
Now, I can run gem5 for linux kernel v4.8.13 and ubuntu 16.04.1 with
kvm support. And the speedup is so amazing. It used to take me 20 ~ 30
minutes to boot up the system without the kvm cpu. Now, it takes only
several seconds!!!
Thanks so much!
best,
Da
Post by Jason Lowe-Power
These patches "fix" the problem. However, they may not apply cleanly to
HEAD and they definitely are not cleanly implemented.
https://gem5-review.googlesource.com/c/public/gem5/+/7362
https://gem5-review.googlesource.com/c/public/gem5/+/7361
Cheers,
Jason
Post by Da Zhang
I am trying to run fs.py with kvm support which might help speedup
our simulation in full system mode. I find the cpu type X86KvmCPU which is
a "kvm-based hardware virtualized cpu". But running fs.py failed with the
panic: KVM: Failed to enter virtualized mode (hw reason: 0x80000021)
Memory Usage: 2416600 KBytes
Program aborted at tick 53418967500
--- BEGIN LIBC BACKTRACE ---
build/X86/gem5.fast(_Z15print_backtracev+0x1f)[0xaee60f]
build/X86/gem5.fast(_Z12abortHandleri+0x34)[0xaee6f4]
/lib64/libpthread.so.0(+0xf5e0)[0x7f5b9ac685e0]
/lib64/libc.so.6(gsignal+0x37)[0x7f5b9901c1f7]
/lib64/libc.so.6(abort+0x148)[0x7f5b9901d8e8]
build/X86/gem5.fast[0x6627df]
build/X86/gem5.fast[0x95d518]
build/X86/gem5.fast(_ZN10BaseKvmCPU13handleKvmExitEv+0x249)[0xc10859]
build/X86/gem5.fast[0xbade3c]
build/X86/gem5.fast(_ZN10EventQueue10serviceOneEv+0x91)[0xad4fc1]
build/X86/gem5.fast(_Z9doSimLoopP10EventQueue+0xa0)[0xb5b110]
build/X86/gem5.fast(_Z8simulatem+0x1f3)[0xb5b563]
build/X86/gem5.fast[0x93867d]
build/X86/gem5.fast[0x939c65]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x730a)[0x7f5b9a56b0ca]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x663c)[0x7f5b9a56a3fc]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x67bd)[0x7f5b9a56a57d]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x67bd)[0x7f5b9a56a57d]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f5b9a56d002]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5513)[0x7f5b9a5692d3]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x663c)[0x7f5b9a56a3fc]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f5b9a56d002]
/lib64/libpython2.7.so.1.0(+0x10043f)[0x7f5b9a58643f]
/lib64/libpython2.7.so.1.0(PyRun_StringFlags+0x65)[0x7f5b9a5872a5]
build/X86/gem5.fast(_Z6m5MainiPPc+0x5f)[0xad327f]
build/X86/gem5.fast(main+0x33)[0x5ddf23]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f5b99008c05]
build/X86/gem5.fast[0x5df8fc]
--- END LIBC BACKTRACE ---
Aborted (core dumped)
build/X86/gem5.fast -d ~/tmp/output1/ configs/example/fs.py
--mem-size=2GB --disk-image=linux-x86.img --cpu-type=X86KvmCPU
Any idea? thanks in advance.
best,
Da
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
Jason Lowe-Power
2018-02-15 17:17:34 UTC
Permalink
Hi Da,

You likely need to enable gem5's multithreaded mode to get many CPUs to
boot correctly. I've had success with up to 32 cores on a 4-core 8-thread
system. I'm not sure if fs.py automatically does this correctly or not. See
my scripts here:
https://github.com/jlpresearch/gem5/tree/jason/kvm-testing/configs/myconfigs
(note:
I haven't rebased in a couple of months).

Cheers,
Jason
Post by Da Zhang
Hey Jason,
The package works.
However, I encountered performance issues with increased CPU number. The
performance was very great with up to 4 CPUs ("-n", I am assuming "number
of CPUs" equal to "number of cores" for testing multithreading workload
later). However, the system fails to boot (or maybe it was just too slow)
when I scale it >= 5 CPUs. Any ideas or suggestions?
10 physic cores each (and 40 logical cores in total). I only used the fs.py
script with --cpu-type=X86KvmCPU, --mem-size=8GB and -n 4. Moreover, the
ancient Linux kernel and image from gem5 website have no performance
problem with increased CPUs.
best,
Da Zhang
Post by Da Zhang
Hi Jason
The package works (I used the second one)! And it also works with the
package you provided (
https://gem5-review.googlesource.com/c/public/gem5/+/7301) in my another
email to fix the keyboard and mouse issue for running later linux kernel
and ubuntu. (However, there are some conflicts in this package, and it is a
little tricky to merge them.)
Now, I can run gem5 for linux kernel v4.8.13 and ubuntu 16.04.1 with
kvm support. And the speedup is so amazing. It used to take me 20 ~ 30
minutes to boot up the system without the kvm cpu. Now, it takes only
several seconds!!!
Thanks so much!
best,
Da
Post by Jason Lowe-Power
These patches "fix" the problem. However, they may not apply cleanly to
HEAD and they definitely are not cleanly implemented.
https://gem5-review.googlesource.com/c/public/gem5/+/7362
https://gem5-review.googlesource.com/c/public/gem5/+/7361
Cheers,
Jason
Post by Da Zhang
I am trying to run fs.py with kvm support which might help speedup
our simulation in full system mode. I find the cpu type X86KvmCPU which is
a "kvm-based hardware virtualized cpu". But running fs.py failed with the
panic: KVM: Failed to enter virtualized mode (hw reason: 0x80000021)
Memory Usage: 2416600 KBytes
Program aborted at tick 53418967500
--- BEGIN LIBC BACKTRACE ---
build/X86/gem5.fast(_Z15print_backtracev+0x1f)[0xaee60f]
build/X86/gem5.fast(_Z12abortHandleri+0x34)[0xaee6f4]
/lib64/libpthread.so.0(+0xf5e0)[0x7f5b9ac685e0]
/lib64/libc.so.6(gsignal+0x37)[0x7f5b9901c1f7]
/lib64/libc.so.6(abort+0x148)[0x7f5b9901d8e8]
build/X86/gem5.fast[0x6627df]
build/X86/gem5.fast[0x95d518]
build/X86/gem5.fast(_ZN10BaseKvmCPU13handleKvmExitEv+0x249)[0xc10859]
build/X86/gem5.fast[0xbade3c]
build/X86/gem5.fast(_ZN10EventQueue10serviceOneEv+0x91)[0xad4fc1]
build/X86/gem5.fast(_Z9doSimLoopP10EventQueue+0xa0)[0xb5b110]
build/X86/gem5.fast(_Z8simulatem+0x1f3)[0xb5b563]
build/X86/gem5.fast[0x93867d]
build/X86/gem5.fast[0x939c65]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x730a)[0x7f5b9a56b0ca]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x663c)[0x7f5b9a56a3fc]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x67bd)[0x7f5b9a56a57d]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x67bd)[0x7f5b9a56a57d]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f5b9a56d002]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5513)[0x7f5b9a5692d3]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x663c)[0x7f5b9a56a3fc]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f5b9a56d002]
/lib64/libpython2.7.so.1.0(+0x10043f)[0x7f5b9a58643f]
/lib64/libpython2.7.so.1.0(PyRun_StringFlags+0x65)[0x7f5b9a5872a5]
build/X86/gem5.fast(_Z6m5MainiPPc+0x5f)[0xad327f]
build/X86/gem5.fast(main+0x33)[0x5ddf23]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f5b99008c05]
build/X86/gem5.fast[0x5df8fc]
--- END LIBC BACKTRACE ---
Aborted (core dumped)
build/X86/gem5.fast -d ~/tmp/output1/ configs/example/fs.py
--mem-size=2GB --disk-image=linux-x86.img --cpu-type=X86KvmCPU
Any idea? thanks in advance.
best,
Da
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
Da Zhang
2018-02-15 21:57:59 UTC
Permalink
Hi Jason,

Thanks a lot! Your scripts work!
I am able to scale up to 40 cores on a compute node with 20 physical cores
/ 40 logical cores with your scripts. So I think the fs.py doesn't do
multithreaded mode correctly for running with kvm cpu. Some of your code
are commented as "required" for running kvm, is that all I need to make
fs.py work?
Moreover, can I use --fast-forward combined with kvm? Our goal is to run a
program with kvm to an interesting point and then switch to a detailed cpu
for experiments. Can we specify the number of instructions to fast forward
(as with --fast-forward option)? So that we can avoid the long wait time
for things like initialization.

best,
Da Zhang
Post by Jason Lowe-Power
Hi Da,
You likely need to enable gem5's multithreaded mode to get many CPUs to
boot correctly. I've had success with up to 32 cores on a 4-core 8-thread
system. I'm not sure if fs.py automatically does this correctly or not. See
my scripts here: https://github.com/jlpresearch/gem5/tree/jason/
kvm-testing/configs/myconfigs (note: I haven't rebased in a couple of
months).
Cheers,
Jason
Post by Da Zhang
Hey Jason,
The package works. However, I encountered performance issues with increased CPU
number. The performance was very great with up to 4 CPUs ("-n", I am
assuming "number of CPUs" equal to "number of cores" for testing
multithreading workload later). However, the system fails to boot (or maybe
it was just too slow) when I scale it >= 5 CPUs. Any ideas or suggestions?
2.40GHz, 10 physic cores each (and 40 logical cores in total). I only used
the fs.py script with --cpu-type=X86KvmCPU, --mem-size=8GB and -n 4.
Moreover, the ancient Linux kernel and image from gem5 website have no
performance problem with increased CPUs.
best,
Da Zhang
Post by Da Zhang
Hi Jason
The package works (I used the second one)! And it also works with the
package you provided (https://gem5-review.googlesource.com/c/public/
gem5/+/7301) in my another email to fix the keyboard and mouse issue
for running later linux kernel and ubuntu. (However, there are some
conflicts in this package, and it is a little tricky to merge them.)
Now, I can run gem5 for linux kernel v4.8.13 and ubuntu 16.04.1 with
kvm support. And the speedup is so amazing. It used to take me 20 ~ 30
minutes to boot up the system without the kvm cpu. Now, it takes only
several seconds!!!
Thanks so much!
best,
Da
Post by Jason Lowe-Power
These patches "fix" the problem. However, they may not apply cleanly to
HEAD and they definitely are not cleanly implemented.
https://gem5-review.googlesource.com/c/public/gem5/+/7362
https://gem5-review.googlesource.com/c/public/gem5/+/7361
Cheers,
Jason
Post by Da Zhang
I am trying to run fs.py with kvm support which might help speedup
our simulation in full system mode. I find the cpu type X86KvmCPU which is
a "kvm-based hardware virtualized cpu". But running fs.py failed with the
panic: KVM: Failed to enter virtualized mode (hw reason: 0x80000021)
Memory Usage: 2416600 KBytes
Program aborted at tick 53418967500
--- BEGIN LIBC BACKTRACE ---
build/X86/gem5.fast(_Z15print_backtracev+0x1f)[0xaee60f]
build/X86/gem5.fast(_Z12abortHandleri+0x34)[0xaee6f4]
/lib64/libpthread.so.0(+0xf5e0)[0x7f5b9ac685e0]
/lib64/libc.so.6(gsignal+0x37)[0x7f5b9901c1f7]
/lib64/libc.so.6(abort+0x148)[0x7f5b9901d8e8]
build/X86/gem5.fast[0x6627df]
build/X86/gem5.fast[0x95d518]
build/X86/gem5.fast(_ZN10BaseKvmCPU13handleKvmExitEv+0x249)[0xc10859]
build/X86/gem5.fast[0xbade3c]
build/X86/gem5.fast(_ZN10EventQueue10serviceOneEv+0x91)[0xad4fc1]
build/X86/gem5.fast(_Z9doSimLoopP10EventQueue+0xa0)[0xb5b110]
build/X86/gem5.fast(_Z8simulatem+0x1f3)[0xb5b563]
build/X86/gem5.fast[0x93867d]
build/X86/gem5.fast[0x939c65]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x730a)[0x7f5b9a56b0ca]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x663c)[0x7f5b9a56a3fc]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x67bd)[0x7f5b9a56a57d]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x67bd)[0x7f5b9a56a57d]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f5b9a56d002]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5513)[0x7f5b9a5692d3]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x663c)[0x7f5b9a56a3fc]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f5b9a56d002]
/lib64/libpython2.7.so.1.0(+0x10043f)[0x7f5b9a58643f]
/lib64/libpython2.7.so.1.0(PyRun_StringFlags+0x65)[0x7f5b9a5872a5]
build/X86/gem5.fast(_Z6m5MainiPPc+0x5f)[0xad327f]
build/X86/gem5.fast(main+0x33)[0x5ddf23]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f5b99008c05]
build/X86/gem5.fast[0x5df8fc]
--- END LIBC BACKTRACE ---
Aborted (core dumped)
build/X86/gem5.fast -d ~/tmp/output1/ configs/example/fs.py
--mem-size=2GB --disk-image=linux-x86.img --cpu-type=X86KvmCPU
Any idea? thanks in advance.
best,
Da
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
Jason Lowe-Power
2018-02-18 21:50:35 UTC
Permalink
Hi Da,

You can look at some of the other scripts in my github for examples of how
to fast forward with KVM.

I suggest that you *not* use fs.py since you know what you want to do. You
should write a script like my runkvm.py that runs gem5 exactly how you
need. fs/se.py were originally created as *examples* of how to write a
runscript. They've grown beyond that, though :).

I don't know exactly what is required and what isn't. You can play around
with the scripts and find out.

Cheers,
Jason
Post by Da Zhang
Hi Jason,
Thanks a lot! Your scripts work!
I am able to scale up to 40 cores on a compute node with 20 physical cores
/ 40 logical cores with your scripts. So I think the fs.py doesn't do
multithreaded mode correctly for running with kvm cpu. Some of your code
are commented as "required" for running kvm, is that all I need to make
fs.py work?
Moreover, can I use --fast-forward combined with kvm? Our goal is to run a
program with kvm to an interesting point and then switch to a detailed cpu
for experiments. Can we specify the number of instructions to fast forward
(as with --fast-forward option)? So that we can avoid the long wait time
for things like initialization.
best,
Da Zhang
Post by Jason Lowe-Power
Hi Da,
You likely need to enable gem5's multithreaded mode to get many CPUs to
boot correctly. I've had success with up to 32 cores on a 4-core 8-thread
system. I'm not sure if fs.py automatically does this correctly or not. See
I haven't rebased in a couple of months).
Cheers,
Jason
Post by Da Zhang
Hey Jason,
The package works.
However, I encountered performance issues with increased CPU number.
The performance was very great with up to 4 CPUs ("-n", I am assuming
"number of CPUs" equal to "number of cores" for testing multithreading
workload later). However, the system fails to boot (or maybe it was just
too slow) when I scale it >= 5 CPUs. Any ideas or suggestions?
2.40GHz, 10 physic cores each (and 40 logical cores in total). I only used
the fs.py script with --cpu-type=X86KvmCPU, --mem-size=8GB and -n 4.
Moreover, the ancient Linux kernel and image from gem5 website have no
performance problem with increased CPUs.
best,
Da Zhang
Post by Da Zhang
Hi Jason
The package works (I used the second one)! And it also works with the
package you provided (
https://gem5-review.googlesource.com/c/public/gem5/+/7301) in my
another email to fix the keyboard and mouse issue for running later
linux kernel and ubuntu. (However, there are some conflicts in this
package, and it is a little tricky to merge them.)
Now, I can run gem5 for linux kernel v4.8.13 and ubuntu 16.04.1 with
kvm support. And the speedup is so amazing. It used to take me 20 ~ 30
minutes to boot up the system without the kvm cpu. Now, it takes only
several seconds!!!
Thanks so much!
best,
Da
Post by Jason Lowe-Power
These patches "fix" the problem. However, they may not apply cleanly
to HEAD and they definitely are not cleanly implemented.
https://gem5-review.googlesource.com/c/public/gem5/+/7362
https://gem5-review.googlesource.com/c/public/gem5/+/7361
Cheers,
Jason
Post by Da Zhang
I am trying to run fs.py with kvm support which might help speedup
our simulation in full system mode. I find the cpu type X86KvmCPU which is
a "kvm-based hardware virtualized cpu". But running fs.py failed with the
panic: KVM: Failed to enter virtualized mode (hw reason: 0x80000021)
Memory Usage: 2416600 KBytes
Program aborted at tick 53418967500
--- BEGIN LIBC BACKTRACE ---
build/X86/gem5.fast(_Z15print_backtracev+0x1f)[0xaee60f]
build/X86/gem5.fast(_Z12abortHandleri+0x34)[0xaee6f4]
/lib64/libpthread.so.0(+0xf5e0)[0x7f5b9ac685e0]
/lib64/libc.so.6(gsignal+0x37)[0x7f5b9901c1f7]
/lib64/libc.so.6(abort+0x148)[0x7f5b9901d8e8]
build/X86/gem5.fast[0x6627df]
build/X86/gem5.fast[0x95d518]
build/X86/gem5.fast(_ZN10BaseKvmCPU13handleKvmExitEv+0x249)[0xc10859]
build/X86/gem5.fast[0xbade3c]
build/X86/gem5.fast(_ZN10EventQueue10serviceOneEv+0x91)[0xad4fc1]
build/X86/gem5.fast(_Z9doSimLoopP10EventQueue+0xa0)[0xb5b110]
build/X86/gem5.fast(_Z8simulatem+0x1f3)[0xb5b563]
build/X86/gem5.fast[0x93867d]
build/X86/gem5.fast[0x939c65]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x730a)[0x7f5b9a56b0ca]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x663c)[0x7f5b9a56a3fc]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x67bd)[0x7f5b9a56a57d]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x67bd)[0x7f5b9a56a57d]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f5b9a56d002]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5513)[0x7f5b9a5692d3]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x663c)[0x7f5b9a56a3fc]
/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed)[0x7f5b9a56cefd]
/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7f5b9a56d002]
/lib64/libpython2.7.so.1.0(+0x10043f)[0x7f5b9a58643f]
/lib64/libpython2.7.so.1.0(PyRun_StringFlags+0x65)[0x7f5b9a5872a5]
build/X86/gem5.fast(_Z6m5MainiPPc+0x5f)[0xad327f]
build/X86/gem5.fast(main+0x33)[0x5ddf23]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f5b99008c05]
build/X86/gem5.fast[0x5df8fc]
--- END LIBC BACKTRACE ---
Aborted (core dumped)
build/X86/gem5.fast -d ~/tmp/output1/ configs/example/fs.py
--mem-size=2GB --disk-image=linux-x86.img --cpu-type=X86KvmCPU
Any idea? thanks in advance.
best,
Da
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
Loading...