[gem5-users] Checkpoint Creation in SE mode

Discussion:

Sakshi Tiwari

2018-08-09 19:23:14 UTC

Hi,

I am running gem5 in se mode and I am using SPEC2006 test suite for my
experiments. To run the representative section of each of the SPEC
benchmark I created simpoints. I have a few questions now:

1) Is it necessary to create checkpoints? Or is there any other way to
simply fast forward the simulation on a multi-core system?

2) If it is necessary to create checkpoints then can someone please confirm
if the following commands are correct. I am able to follow step (a) and (b)
but most of the SPEC benchmarks are failing when I run step (c).
(a) Create checkpoint for each of the simpoint:
build/X86/gem5.opt configs/example/se.py
--take-simpoint-checkpoint=m5out/simpoints/libq/libq.simpoint,m5out/simpoints/libq/libq.weight,3000000,10000000
--cmd="benchmarks/ben/libquantum_base.amd64-m64-gcc43-nn" --options="1397
8" --cpu-type=DerivO3CPU --l1d_size=32kB --l1i_size=32kB --l2_size=256kB
--caches --l2cache --l3cache --l3_size=8MB --num-cpus=1 --mem-size=8192MB
(b) Run checkpoint_aggregator script to aggregate the checkpoints which
would run on 4 different cores (I am running the same checkpoint on all the
4 cores):
python util/checkpoint_aggregator.py -o
/home/sakshi/workspace2/gem5-63325e5b0a9d-modified/m5out/libq_cpt/cpt.None.4
-c --cpts
/home/sakshi/workspace2/gem5-63325e5b0a9d-modified/m5out/libq_cpt/cpt.simpoint_03_inst_1100000000_weight_0.069900_interval_3000000_warmup_10000000/
/home/sakshi/workspace2/gem5-63325e5b0a9d-modified/m5out/libq_cpt/cpt.simpoint_03_inst_1100000000_weight_0.069900_interval_3000000_warmup_10000000/
/home/sakshi/workspace2/gem5-63325e5b0a9d-modified/m5out/libq_cpt/cpt.simpoint_03_inst_1100000000_weight_0.069900_interval_3000000_warmup_10000000/
/home/sakshi/workspace2/gem5-63325e5b0a9d-modified/m5out/libq_cpt/cpt.simpoint_03_inst_1100000000_weight_0.069900_interval_3000000_warmup_10000000/
--memory-size 8589934592
(c) Run the simulation in atomic mode and restore it to the normal mode
(i.e. Timing+O3) from the aggregated checkpoint:
build/X86/gem5.opt configs/example/se.py --at-instruction
--checkpoint-restore=4 --restore-with-cpu=detailed --checkpoint-dir
/home/sakshi/workspace2/gem5-63325e5b0a9d-modified/m5out/libq_cpt/
--cmd="benchmarks/ben/libquantum_base.amd64-m64-gcc43-nn;benchmarks/ben/libquantum_base.amd64-m64-gcc43-nn;benchmarks/ben/libquantum_base.amd64-m64-gcc43-nn;benchmarks/ben/libquantum_base.amd64-m64-gcc43-nn"
--options="1397 8;1397 8;1397 8;1397 8" --cpu-type=DerivO3CPU
--l1d_size=32kB --l1i_size=32kB --l2_size=256kB --caches --l2cache
--l3cache --l3_size=8MB --num-cpus=4 --mem-size=8192MB -I 100000000

Regards,
Sakshi

Sakshi Tiwari

2018-08-22 12:53:57 UTC

Permalink

Can someone please reply to the query above? It will be really helpful.

Regards,
Sakshi

Post by Sakshi Tiwari
Hi,
I am running gem5 in se mode and I am using SPEC2006 test suite for my
experiments. To run the representative section of each of the SPEC
1) Is it necessary to create checkpoints? Or is there any other way to
simply fast forward the simulation on a multi-core system?
2) If it is necessary to create checkpoints then can someone please
confirm if the following commands are correct. I am able to follow step (a)
and (b) but most of the SPEC benchmarks are failing when I run step (c).
build/X86/gem5.opt configs/example/se.py
--take-simpoint-checkpoint=m5out/simpoints/libq/libq.simpoint,m5out/simpoints/libq/libq.weight,3000000,10000000
--cmd="benchmarks/ben/libquantum_base.amd64-m64-gcc43-nn" --options="1397
8" --cpu-type=DerivO3CPU --l1d_size=32kB --l1i_size=32kB --l2_size=256kB
--caches --l2cache --l3cache --l3_size=8MB --num-cpus=1 --mem-size=8192MB
(b) Run checkpoint_aggregator script to aggregate the checkpoints which
would run on 4 different cores (I am running the same checkpoint on all the
python util/checkpoint_aggregator.py -o
/home/sakshi/workspace2/gem5-63325e5b0a9d-modified/m5out/libq_cpt/cpt.None.4
-c --cpts
/home/sakshi/workspace2/gem5-63325e5b0a9d-modified/m5out/libq_cpt/cpt.simpoint_03_inst_1100000000_weight_0.069900_interval_3000000_warmup_10000000/
/home/sakshi/workspace2/gem5-63325e5b0a9d-modified/m5out/libq_cpt/cpt.simpoint_03_inst_1100000000_weight_0.069900_interval_3000000_warmup_10000000/
/home/sakshi/workspace2/gem5-63325e5b0a9d-modified/m5out/libq_cpt/cpt.simpoint_03_inst_1100000000_weight_0.069900_interval_3000000_warmup_10000000/
/home/sakshi/workspace2/gem5-63325e5b0a9d-modified/m5out/libq_cpt/cpt.simpoint_03_inst_1100000000_weight_0.069900_interval_3000000_warmup_10000000/
--memory-size 8589934592
(c) Run the simulation in atomic mode and restore it to the normal mode
build/X86/gem5.opt configs/example/se.py --at-instruction
--checkpoint-restore=4 --restore-with-cpu=detailed --checkpoint-dir
/home/sakshi/workspace2/gem5-63325e5b0a9d-modified/m5out/libq_cpt/
--cmd="benchmarks/ben/libquantum_base.amd64-m64-gcc43-nn;benchmarks/ben/libquantum_base.amd64-m64-gcc43-nn;benchmarks/ben/libquantum_base.amd64-m64-gcc43-nn;benchmarks/ben/libquantum_base.amd64-m64-gcc43-nn"
--options="1397 8;1397 8;1397 8;1397 8" --cpu-type=DerivO3CPU
--l1d_size=32kB --l1i_size=32kB --l2_size=256kB --caches --l2cache
--l3cache --l3_size=8MB --num-cpus=4 --mem-size=8192MB -I 100000000
Regards,
Sakshi

Abhishek Singh

2018-08-22 13:23:03 UTC

Permalink

You can also use simpoints else just fast forward the start up instruction
and run for some instruction using -I

Post by Sakshi Tiwari
Can someone please reply to the query above? It will be really helpful.
Regards,
Sakshi

_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users

Timon Evenblij

2018-08-23 07:28:25 UTC

Permalink

Hi Sakshi,

I did not check completely, but at first sight you are misinterpreting the
--restore-with-cpu option.
https://stackoverflow.com/questions/49011096/how-to-switch-cpu-models-in-gem5-after-restoring-a-checkpoint-and-then-observe-t
has information on what it does: use --restore-with-cpu only if you made a
checkpoint with a non-atomic cpu model. Otherwise, don't use this option
and just say cpu-type=detailed or whatever model you want to use after
resuming the checkpoint.

The reasoning is that gem5 needs to know the exact model which the
checkpoint was made with (normally this is the atomic model). It will
resume the checkpoint using the --restore-with-cpu option (atomic by
default) to set all the state correctly. After setting up the state, it
will switch to the model specified with the regular --cpu-type option, to
do the intended simulation. I have to admit it is a bit confusing, but
hopefully you understand it now.

Regards

Timon

On Wed, Aug 22, 2018 at 3:23 PM Abhishek Singh <

Post by Abhishek Singh
You can also use simpoints else just fast forward the start up instruction
and run for some instruction using -I

Post by Sakshi Tiwari
Can someone please reply to the query above? It will be really helpful.
Regards,
Sakshi

_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users

Sakshi Tiwari

2018-08-23 18:10:47 UTC

Permalink

Hi Timon,
Thank you to point this out.

Post by Timon Evenblij
Hi Sakshi,
I did not check completely, but at first sight you are misinterpreting the
--restore-with-cpu option.
https://stackoverflow.com/questions/49011096/how-to-switch-cpu-models-in-gem5-after-restoring-a-checkpoint-and-then-observe-t
has information on what it does: use --restore-with-cpu only if you made a
checkpoint with a non-atomic cpu model. Otherwise, don't use this option
and just say cpu-type=detailed or whatever model you want to use after
resuming the checkpoint.
The reasoning is that gem5 needs to know the exact model which the
checkpoint was made with (normally this is the atomic model). It will
resume the checkpoint using the --restore-with-cpu option (atomic by
default) to set all the state correctly. After setting up the state, it
will switch to the model specified with the regular --cpu-type option, to
do the intended simulation. I have to admit it is a bit confusing, but
hopefully you understand it now.
Regards
Timon
On Wed, Aug 22, 2018 at 3:23 PM Abhishek Singh <

Post by Abhishek Singh
You can also use simpoints else just fast forward the start up
instruction and run for some instruction using -I

Post by Sakshi Tiwari
Can someone please reply to the query above? It will be really helpful.
Regards,
Sakshi

Post by Sakshi Tiwari
Hi,
I am running gem5 in se mode and I am using SPEC2006 test suite for my
experiments. To run the representative section of each of the SPEC
1) Is it necessary to create checkpoints? Or is there any other way to
simply fast forward the simulation on a multi-core system?
2) If it is necessary to create checkpoints then can someone please
confirm if the following commands are correct. I am able to follow step (a)
and (b) but most of the SPEC benchmarks are failing when I run step (c).
build/X86/gem5.opt configs/example/se.py
--take-simpoint-checkpoint=m5out/simpoints/libq/libq.simpoint,m5out/simpoints/libq/libq.weight,3000000,10000000
--cmd="benchmarks/ben/libquantum_base.amd64-m64-gcc43-nn" --options="1397
8" --cpu-type=DerivO3CPU --l1d_size=32kB --l1i_size=32kB --l2_size=256kB
--caches --l2cache --l3cache --l3_size=8MB --num-cpus=1 --mem-size=8192MB
(b) Run checkpoint_aggregator script to aggregate the checkpoints
which would run on 4 different cores (I am running the same checkpoint on
python util/checkpoint_aggregator.py -o
/home/sakshi/workspace2/gem5-63325e5b0a9d-modified/m5out/libq_cpt/cpt.None.4
-c --cpts
/home/sakshi/workspace2/gem5-63325e5b0a9d-modified/m5out/libq_cpt/cpt.simpoint_03_inst_1100000000_weight_0.069900_interval_3000000_warmup_10000000/
/home/sakshi/workspace2/gem5-63325e5b0a9d-modified/m5out/libq_cpt/cpt.simpoint_03_inst_1100000000_weight_0.069900_interval_3000000_warmup_10000000/
/home/sakshi/workspace2/gem5-63325e5b0a9d-modified/m5out/libq_cpt/cpt.simpoint_03_inst_1100000000_weight_0.069900_interval_3000000_warmup_10000000/
/home/sakshi/workspace2/gem5-63325e5b0a9d-modified/m5out/libq_cpt/cpt.simpoint_03_inst_1100000000_weight_0.069900_interval_3000000_warmup_10000000/
--memory-size 8589934592
(c) Run the simulation in atomic mode and restore it to the normal mode
build/X86/gem5.opt configs/example/se.py --at-instruction
--checkpoint-restore=4 --restore-with-cpu=detailed --checkpoint-dir
/home/sakshi/workspace2/gem5-63325e5b0a9d-modified/m5out/libq_cpt/
--cmd="benchmarks/ben/libquantum_base.amd64-m64-gcc43-nn;benchmarks/ben/libquantum_base.amd64-m64-gcc43-nn;benchmarks/ben/libquantum_base.amd64-m64-gcc43-nn;benchmarks/ben/libquantum_base.amd64-m64-gcc43-nn"
--options="1397 8;1397 8;1397 8;1397 8" --cpu-type=DerivO3CPU
--l1d_size=32kB --l1i_size=32kB --l2_size=256kB --caches --l2cache
--l3cache --l3_size=8MB --num-cpus=4 --mem-size=8192MB -I 100000000
Regards,
Sakshi

_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users

Sakshi Tiwari

2018-08-23 15:14:36 UTC

Permalink

Thank you, Abhishek.
I observed an unexpected behavior while fast forwarding. I fast forwarded
the simulation for 1 billion instructions and then simulated for 100
million instructions (using -I). However, in the stats.txt, total simulated
instructions are approx 2 billions (because 2 benchmarks are running) and
simulated instruction for every cpu and switch_cpus is 0. Do you have any
idea why is this happening?

Just in case if it is important, I used the following command:
build/X86/gem5.opt configs/example/se.py
--cmd="/home/sakshi/benchmarks/ben/mcf_base.amd64-m64-gcc43-nn;/home/sakshi/benchmarks/ben/mcf_base.amd64-m64-gcc43-nn"
--options="/home/sakshi/benchmarks/ben/inp.in;/home/sakshi/benchmarks/ben/
inp.in" --cpu-type=DerivO3CPU --l1d_size=32kB --l1i_size=32kB
--l2_size=256kB --caches --l2cache --l3cache --l3_size=8MB --num-cpus=2
--mem-size=8192MB --fast-forward=1000000000 -I 100000000

On Wed, Aug 22, 2018 at 6:53 PM Abhishek Singh <

Post by Abhishek Singh
You can also use simpoints else just fast forward the start up instruction
and run for some instruction using -I

Post by Sakshi Tiwari
Can someone please reply to the query above? It will be really helpful.
Regards,
Sakshi

_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users