Discussion:
Simulating multiprogrammed & multithreaded workloads in SE mode?
(too old to reply)
Majid Namaki Shoushtari
2016-11-18 07:48:01 UTC
Permalink
I was wondering if there is a way to simulate multiprogrammed/multithreaded
workloads in SE mode?
Just to clarify: I mean running for example two programs that spawn 4
threads each.

As I understand this is not possible with the current se.py script, but is
it theoretically possible to run such simulations in SE mode?
If yes, could you please give some hints on how to modify se.py accordingly?

Thanks,
Majid
Jason Lowe-Power
2016-11-20 20:21:43 UTC
Permalink
Hi Majid,

Yes. This is possible. To run multiple different processes, you can create
multiple CPUs with multiple "LiveProcess" objects. For multi-threaded
applications, I believe you need to have enough CPUs create (enough
hardware contexts) and then link to m5threads instead of the pthreads
library.

Hopefully this will get your started down the right path.

Jason
Post by Majid Namaki Shoushtari
I was wondering if there is a way to simulate
multiprogrammed/multithreaded workloads in SE mode?
Just to clarify: I mean running for example two programs that spawn 4
threads each.
As I understand this is not possible with the current se.py script, but is
it theoretically possible to run such simulations in SE mode?
If yes, could you please give some hints on how to modify se.py accordingly?
Thanks,
Majid
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
Gutierrez, Anthony
2016-11-20 20:39:22 UTC
Permalink
I do not believe m5threads is necessary any longer, Brandon Potter’s work on adding dynamic linker/loader support to gem5 allows for you to dynamically load the host’s pthreads library. I also believe, although Brandon can correct me if I am wrong, that with some of the patches he currently has on RB that you no longer need to provision for the number of threads that you will need prior to running. I believe his patches for clone/exec allow this to happen dynamically.

-Tony

From: gem5-users [mailto:gem5-users-***@gem5.org] On Behalf Of Jason Lowe-Power
Sent: Sunday, November 20, 2016 12:22 PM
To: gem5 users mailing list <gem5-***@gem5.org>
Subject: Re: [gem5-users] Simulating multiprogrammed & multithreaded workloads in SE mode?

Hi Majid,

Yes. This is possible. To run multiple different processes, you can create multiple CPUs with multiple "LiveProcess" objects. For multi-threaded applications, I believe you need to have enough CPUs create (enough hardware contexts) and then link to m5threads instead of the pthreads library.

Hopefully this will get your started down the right path.

Jason

On Fri, Nov 18, 2016 at 1:48 AM Majid Namaki Shoushtari <***@uci.edu<mailto:***@uci.edu>> wrote:
I was wondering if there is a way to simulate multiprogrammed/multithreaded workloads in SE mode?
Just to clarify: I mean running for example two programs that spawn 4 threads each.

As I understand this is not possible with the current se.py script, but is it theoretically possible to run such simulations in SE mode?
If yes, could you please give some hints on how to modify se.py accordingly?

Thanks,
Majid
_______________________________________________
gem5-users mailing list
gem5-***@gem5.org<mailto:gem5-***@gem5.org>
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
Majid Namaki Shoushtari
2016-11-29 03:32:48 UTC
Permalink
Hi Tony,

Thanks for the information. I did see Brandon's presentation on NPTL in
last year's gem5 user workshop, but looking at the commit history, I didn't
see his patch being added to dev branch. Any pointer on that?

Best,
Majid

On Sun, Nov 20, 2016 at 12:39 PM, Gutierrez, Anthony <
Post by Gutierrez, Anthony
I do not believe m5threads is necessary any longer, Brandon Potter’s work
on adding dynamic linker/loader support to gem5 allows for you to
dynamically load the host’s pthreads library. I also believe, although
Brandon can correct me if I am wrong, that with some of the patches he
currently has on RB that you no longer need to provision for the number of
threads that you will need prior to running. I believe his patches for
clone/exec allow this to happen dynamically.
-Tony
Lowe-Power
*Sent:* Sunday, November 20, 2016 12:22 PM
*Subject:* Re: [gem5-users] Simulating multiprogrammed & multithreaded
workloads in SE mode?
Hi Majid,
Yes. This is possible. To run multiple different processes, you can create
multiple CPUs with multiple "LiveProcess" objects. For multi-threaded
applications, I believe you need to have enough CPUs create (enough
hardware contexts) and then link to m5threads instead of the pthreads
library.
Hopefully this will get your started down the right path.
Jason
I was wondering if there is a way to simulate
multiprogrammed/multithreaded workloads in SE mode?
Just to clarify: I mean running for example two programs that spawn 4 threads each.
As I understand this is not possible with the current se.py script, but is
it theoretically possible to run such simulations in SE mode?
If yes, could you please give some hints on how to modify se.py accordingly?
Thanks,
Majid
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
--
Majid Namaki Shoushtari
PhD Candidate
Department of Computer Science
University of California, Irvine
Irvine, CA 92697
***@uci.edu
http://www.ics.uci.edu/~anamakis
Majid Namaki Shoushtari
2016-11-21 08:26:49 UTC
Permalink
​
Thanks Jason
​.​


I think I should have been more specific in my question:
I understand how "LiveProcess" objects are created for each application and
how each single threaded workload is assigned to a
​
CPU. The loop in se.py that does the workload to
​
CPU assignments either expects the
​number​
of specified applications to be equal to the number of cores, or expects
only one application. In case of only one
​
application and if #CPUs
​
Post by Jason Lowe-Power
1
​,
it assigns the same app
​​
to all cores
​ (line 237)​
.

To be clear this is the code excerpt I'm referring to:

​​
===============================
233 for i in xrange(np):
234 if options.smt:
235 system.cpu[i].workload = multiprocesses
236 elif len(multiprocesses) == 1:
237 system.cpu[i].workload = multiprocesses[0]
238 else:
239 system.cpu[i].workload = multiprocesses[i]
​
===============================

First of all, how is line 237 not causing that np copies of
multiprocesses[0]
​ to run​ but instead just 1 runs
?

I'm trying to understand how I can create enough hardware context without
assigning a workload to all of them. I modified the loop to run just for
the number of given applications by replacing "for i in xrange(np):" with
"for i in xrange(len(multiprocesses)):", but it throws an error
​ which I believe means I should somehow assign a workload to all CPUs​
:

​
===============================
fatal: Must specify at least one workload!
@ tick 0
[create:build/ARM/cpu/o3/deriv.cc, line 48]
​
===============================

Hope my question is clear.

Thank
​
you
​
for your time.
Majid
Post by Jason Lowe-Power
Hi Majid,
Yes. This is possible. To run multiple different processes, you can create
multiple CPUs with multiple "LiveProcess" objects. For multi-threaded
applications, I believe you need to have enough CPUs create (enough
hardware contexts) and then link to m5threads instead of the pthreads
library.
Hopefully this will get your started down the right path.
Jason
Post by Majid Namaki Shoushtari
I was wondering if there is a way to simulate
multiprogrammed/multithreaded workloads in SE mode?
Just to clarify: I mean running for example two programs that spawn 4
threads each.
As I understand this is not possible with the current se.py script, but
is it theoretically possible to run such simulations in SE mode?
If yes, could you please give some hints on how to modify se.py accordingly?
Thanks,
Majid
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
--
Majid Namaki Shoushtari
PhD Candidate
Department of Computer Science
University of California, Irvine
Irvine, CA 92697
***@uci.edu
http://www.ics.uci.edu/~anamakis
Majid Namaki Shoushtari
2016-11-21 09:11:23 UTC
Permalink
Sorry, somehow the formatting got messed up. I'm sending my last message
again...

================================================================

Thanks Jason​.​

I think I should have been more specific in my question:
I understand how "LiveProcess" objects are created for each application and
how each single threaded workload is assigned to a CPU. The loop in se.py
that does the workload to CPU assignments either expects the ​number of
specified applications to be equal to the number of cores, or expects only
one application. In case of only one application and if #CPUs > 1 , it
assigns the same app to all cores (line 237)​.

To be clear this is the code excerpt I'm referring to:
​​
===============================
233 for i in xrange(np):
234 if options.smt:
235 system.cpu[i].workload = multiprocesses
236 elif len(multiprocesses) == 1:
237 system.cpu[i].workload = multiprocesses[0]
238 else:
239 system.cpu[i].workload = multiprocesses[i]
​===============================

First of all, how is line 237 not causing that np copies of
multiprocesses[0] to run​ but instead just 1 runs ?

I'm trying to understand how I can create enough hardware context without
assigning a workload to all of them. I modified the loop to run just for
the number of given applications by replacing "for i in xrange(np):" with
"for i in xrange(len(multiprocesses)):", but it throws an error which I
believe means I should somehow assign a workload to all CPUs​ :
​
===============================
fatal: Must specify at least one workload!
@ tick 0
[create:build/ARM/cpu/o3/deriv.cc, line 48]
===============================

Hope my question is clear.

Thank you for your time.
Majid
Post by Majid Namaki Shoushtari
​
Thanks Jason
​.​
I understand how "LiveProcess" objects are created for each application
and how each single threaded workload is assigned to a
​
CPU. The loop in se.py that does the workload to
​
CPU assignments either expects the
​number​
of specified applications to be equal to the number of cores, or expects
only one application. In case of only one
​
application and if #CPUs
​
Post by Jason Lowe-Power
1
​,
it assigns the same app
​​
to all cores
​ (line 237)​
.
​​
===============================
235 system.cpu[i].workload = multiprocesses
237 system.cpu[i].workload = multiprocesses[0]
239 system.cpu[i].workload = multiprocesses[i]
​
===============================
First of all, how is line 237 not causing that np copies of
multiprocesses[0]
​ to run​ but instead just 1 runs
?
I'm trying to understand how I can create enough hardware context without
assigning a workload to all of them. I modified the loop to run just for
the number of given applications by replacing "for i in xrange(np):" with
"for i in xrange(len(multiprocesses)):", but it throws an error
​ which I believe means I should somehow assign a workload to all CPUs​
​
===============================
fatal: Must specify at least one workload!
@ tick 0
[create:build/ARM/cpu/o3/deriv.cc, line 48]
​
===============================
Hope my question is clear.
Thank
​
you
​
for your time.
Majid
Post by Jason Lowe-Power
Hi Majid,
Yes. This is possible. To run multiple different processes, you can
create multiple CPUs with multiple "LiveProcess" objects. For
multi-threaded applications, I believe you need to have enough CPUs create
(enough hardware contexts) and then link to m5threads instead of the
pthreads library.
Hopefully this will get your started down the right path.
Jason
Post by Majid Namaki Shoushtari
I was wondering if there is a way to simulate
multiprogrammed/multithreaded workloads in SE mode?
Just to clarify: I mean running for example two programs that spawn 4
threads each.
As I understand this is not possible with the current se.py script, but
is it theoretically possible to run such simulations in SE mode?
If yes, could you please give some hints on how to modify se.py accordingly?
Thanks,
Majid
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
--
Majid Namaki Shoushtari
PhD Candidate
Department of Computer Science
University of California, Irvine
Irvine, CA 92697
http://www.ics.uci.edu/~anamakis
--
Majid Namaki Shoushtari
PhD Candidate
Department of Computer Science
University of California, Irvine
Irvine, CA 92697
***@uci.edu
http://www.ics.uci.edu/~anamakis
Jason Lowe-Power
2016-11-21 15:19:30 UTC
Permalink
Hi Majid,

To answer your question: I'm not sure. I've never tried to use the SMT
support.

Some things I would do to try to solve the problem:
1) Feel free to modify the config scripts to suit your needs! There is no
reason for you to keep that if statement as it is if it isn't doing what
you need it to do.
2) Read the code for how workloads get initialized in the file src/
cpu/o3/deriv.cc (probably starting around line 48) since this is where the
error is occuring.

Hopefully this helps.

Jason
Post by Majid Namaki Shoushtari
Sorry, somehow the formatting got messed up. I'm sending my last message
again...
================================================================
Thanks Jason​.​
I understand how "LiveProcess" objects are created for each application
and how each single threaded workload is assigned to a CPU. The loop in
se.py that does the workload to CPU assignments either expects the ​number
of specified applications to be equal to the number of cores, or expects
only one application. In case of only one application and if #CPUs > 1 , it
assigns the same app to all cores (line 237)​.
​​
===============================
235 system.cpu[i].workload = multiprocesses
237 system.cpu[i].workload = multiprocesses[0]
239 system.cpu[i].workload = multiprocesses[i]
​===============================
First of all, how is line 237 not causing that np copies of
multiprocesses[0] to run​ but instead just 1 runs ?
I'm trying to understand how I can create enough hardware context without
assigning a workload to all of them. I modified the loop to run just for
the number of given applications by replacing "for i in xrange(np):" with
"for i in xrange(len(multiprocesses)):", but it throws an error which I
​
===============================
fatal: Must specify at least one workload!
@ tick 0
[create:build/ARM/cpu/o3/deriv.cc, line 48]
===============================
Hope my question is clear.
Thank you for your time.
Majid
On Mon, Nov 21, 2016 at 12:26 AM, Majid Namaki Shoushtari <
​
Thanks Jason
​.​
I understand how "LiveProcess" objects are created for each application
and how each single threaded workload is assigned to a
​
CPU. The loop in se.py that does the workload to
​
CPU assignments either expects the
​number​
of specified applications to be equal to the number of cores, or expects
only one application. In case of only one
​
application and if #CPUs
​
1
​,
it assigns the same app
​​
to all cores
​ (line 237)​
.
​​
===============================
235 system.cpu[i].workload = multiprocesses
237 system.cpu[i].workload = multiprocesses[0]
239 system.cpu[i].workload = multiprocesses[i]
​
===============================
First of all, how is line 237 not causing that np copies of
multiprocesses[0]
​ to run​ but instead just 1 runs
?
I'm trying to understand how I can create enough hardware context without
assigning a workload to all of them. I modified the loop to run just for
the number of given applications by replacing "for i in xrange(np):" with
"for i in xrange(len(multiprocesses)):", but it throws an error
​ which I believe means I should somehow assign a workload to all CPUs​
​
===============================
fatal: Must specify at least one workload!
@ tick 0
[create:build/ARM/cpu/o3/deriv.cc, line 48]
​
===============================
Hope my question is clear.
Thank
​
you
​
for your time.
Majid
Hi Majid,
Yes. This is possible. To run multiple different processes, you can create
multiple CPUs with multiple "LiveProcess" objects. For multi-threaded
applications, I believe you need to have enough CPUs create (enough
hardware contexts) and then link to m5threads instead of the pthreads
library.
Hopefully this will get your started down the right path.
Jason
I was wondering if there is a way to simulate
multiprogrammed/multithreaded workloads in SE mode?
Just to clarify: I mean running for example two programs that spawn 4
threads each.
As I understand this is not possible with the current se.py script, but is
it theoretically possible to run such simulations in SE mode?
If yes, could you please give some hints on how to modify se.py accordingly?
Thanks,
Majid
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
--
Majid Namaki Shoushtari
PhD Candidate
Department of Computer Science
University of California, Irvine
Irvine, CA 92697
http://www.ics.uci.edu/~anamakis
--
Majid Namaki Shoushtari
PhD Candidate
Department of Computer Science
University of California, Irvine
Irvine, CA 92697
http://www.ics.uci.edu/~anamakis
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
Loading...