Discussion:
bytesWritten < (8 * number of 64-bit stores to unique addresses)
(too old to reply)
Patrick
2015-04-15 18:13:39 UTC
Permalink
I am looking at stats.txt for the amount of data written to the DRAM during
the execution of a process in full system mode. I looked at the execution
trace, and there are at least 512 64-bit stores to unique addresses.
However, stats.txt reports only 2,304 "bytesWritten" to the memory. It is a
4-channel memory configuration. "stats.txt" reports 1,152 "bytesWritten" on
channel 0, 0 "bytesWritten" to channel 1, 0 "bytesWritten" to channel 2,
and 1,152 "bytesWritten" to channel 3.

Does anyone know what would cause this? I thought maybe the data might be
getting left in the caches, but I am waiting until the process exits before
calling m5 resetstats. The "bytesReadDRAM" is less than expected, also,
based on the number of loads in the instruction trace. I thought perhaps
this was because no-write allocate was being used, but the discussion
linked below suggests that default is to use write-allocate. I can't find
where this is configured in gem5, so I'm not able to check this at the
moment.

http://comments.gmane.org/gmane.comp.emulators.m5.users/12597

Any help is appreciated.

​-​
Patrick
Andreas Hansson
2015-04-15 23:26:22 UTC
Permalink
Hi Patrick,

When it comes to the stores you are looking at a rather small number of operations, and my guess is that they are still in the DRAM write queues. These queues are not drained at the moment once the writes fall below the “low water mark”.

Andreas

From: Patrick <***@gmail.com<mailto:***@gmail.com>>
Reply-To: gem5 users mailing list <gem5-***@gem5.org<mailto:gem5-***@gem5.org>>
Date: Wednesday, 15 April 2015 19:13
To: gem5 users mailing list <gem5-***@gem5.org<mailto:gem5-***@gem5.org>>
Subject: [gem5-users] bytesWritten < (8 * number of 64-bit stores to unique addresses)

I am looking at stats.txt for the amount of data written to the DRAM during the execution of a process in full system mode. I looked at the execution trace, and there are at least 512 64-bit stores to unique addresses. However, stats.txt reports only 2,304 "bytesWritten" to the memory. It is a 4-channel memory configuration. "stats.txt" reports 1,152 "bytesWritten" on channel 0, 0 "bytesWritten" to channel 1, 0 "bytesWritten" to channel 2, and 1,152 "bytesWritten" to channel 3.

Does anyone know what would cause this? I thought maybe the data might be getting left in the caches, but I am waiting until the process exits before calling m5 resetstats. The "bytesReadDRAM" is less than expected, also, based on the number of loads in the instruction trace. I thought perhaps this was because no-write allocate was being used, but the discussion linked below suggests that default is to use write-allocate. I can't find where this is configured in gem5, so I'm not able to check this at the moment.

http://comments.gmane.org/gmane.comp.emulators.m5.users/12597

Any help is appreciated.

​-​
Patrick

-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No: 2557590
ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No: 2548782
Patrick
2015-04-16 20:07:02 UTC
Permalink
Thanks, Andreas. This makes sense.
Post by Andreas Hansson
Hi Patrick,
When it comes to the stores you are looking at a rather small number of
operations, and my guess is that they are still in the DRAM write queues.
These queues are not drained at the moment once the writes fall below the
“low water mark”.
Andreas
Date: Wednesday, 15 April 2015 19:13
Subject: [gem5-users] bytesWritten < (8 * number of 64-bit stores to unique addresses)
I am looking at stats.txt for the amount of data written to the DRAM
during the execution of a process in full system mode. I looked at the
execution trace, and there are at least 512 64-bit stores to unique
addresses. However, stats.txt reports only 2,304 "bytesWritten" to the
memory. It is a 4-channel memory configuration. "stats.txt" reports 1,152
"bytesWritten" on channel 0, 0 "bytesWritten" to channel 1, 0
"bytesWritten" to channel 2, and 1,152 "bytesWritten" to channel 3.
Does anyone know what would cause this? I thought maybe the data might
be getting left in the caches, but I am waiting until the process exits
before calling m5 resetstats. The "bytesReadDRAM" is less than expected,
also, based on the number of loads in the instruction trace. I thought
perhaps this was because no-write allocate was being used, but the
discussion linked below suggests that default is to use write-allocate. I
can't find where this is configured in gem5, so I'm not able to check this
at the moment.
http://comments.gmane.org/gmane.comp.emulators.m5.users/12597
Any help is appreciated.
​-​
Patrick
-- IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ,
Registered in England & Wales, Company No: 2557590
ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ,
Registered in England & Wales, Company No: 2548782
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
Patrick
2015-04-21 22:47:50 UTC
Permalink
I looked back at this, and I'm still not sure it's clear to me what is
going on. I decreased the size of the write queue to 2, and when running
the simulation described in my previous message (in which 512 64-bit stores
to unique addresses are issued), bytesWritten in one run was reported to be
only 1,664 bytes. With the write queue set to size 2, I would expect
bytesWritten to be at least 4096 - 128 = 3,968 bytes (the burstSize is 64
bytes).

Any additional help is appreciated.

Regards,
Patrick
Post by Patrick
Thanks, Andreas. This makes sense.
Post by Andreas Hansson
Hi Patrick,
When it comes to the stores you are looking at a rather small number of
operations, and my guess is that they are still in the DRAM write queues.
These queues are not drained at the moment once the writes fall below the
“low water mark”.
Andreas
Date: Wednesday, 15 April 2015 19:13
Subject: [gem5-users] bytesWritten < (8 * number of 64-bit stores to unique addresses)
I am looking at stats.txt for the amount of data written to the DRAM
during the execution of a process in full system mode. I looked at the
execution trace, and there are at least 512 64-bit stores to unique
addresses. However, stats.txt reports only 2,304 "bytesWritten" to the
memory. It is a 4-channel memory configuration. "stats.txt" reports 1,152
"bytesWritten" on channel 0, 0 "bytesWritten" to channel 1, 0
"bytesWritten" to channel 2, and 1,152 "bytesWritten" to channel 3.
Does anyone know what would cause this? I thought maybe the data might
be getting left in the caches, but I am waiting until the process exits
before calling m5 resetstats. The "bytesReadDRAM" is less than expected,
also, based on the number of loads in the instruction trace. I thought
perhaps this was because no-write allocate was being used, but the
discussion linked below suggests that default is to use write-allocate. I
can't find where this is configured in gem5, so I'm not able to check this
at the moment.
http://comments.gmane.org/gmane.comp.emulators.m5.users/12597
Any help is appreciated.
​-​
Patrick
-- IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ,
Registered in England & Wales, Company No: 2557590
ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ,
Registered in England & Wales, Company No: 2548782
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
Prathap Kolakkampadath
2015-04-22 03:27:14 UTC
Permalink
Hello Patrick,

Can you check the number of last level cache misses as reported by
stats.text?

Prathap
Post by Patrick
I looked back at this, and I'm still not sure it's clear to me what is
going on. I decreased the size of the write queue to 2, and when running
the simulation described in my previous message (in which 512 64-bit stores
to unique addresses are issued), bytesWritten in one run was reported to be
only 1,664 bytes. With the write queue set to size 2, I would expect
bytesWritten to be at least 4096 - 128 = 3,968 bytes (the burstSize is 64
bytes).
Any additional help is appreciated.
Regards,
Patrick
Post by Patrick
Thanks, Andreas. This makes sense.
Post by Andreas Hansson
Hi Patrick,
When it comes to the stores you are looking at a rather small number
of operations, and my guess is that they are still in the DRAM write
queues. These queues are not drained at the moment once the writes fall
below the “low water mark”.
Andreas
Date: Wednesday, 15 April 2015 19:13
Subject: [gem5-users] bytesWritten < (8 * number of 64-bit stores to unique addresses)
I am looking at stats.txt for the amount of data written to the DRAM
during the execution of a process in full system mode. I looked at the
execution trace, and there are at least 512 64-bit stores to unique
addresses. However, stats.txt reports only 2,304 "bytesWritten" to the
memory. It is a 4-channel memory configuration. "stats.txt" reports 1,152
"bytesWritten" on channel 0, 0 "bytesWritten" to channel 1, 0
"bytesWritten" to channel 2, and 1,152 "bytesWritten" to channel 3.
Does anyone know what would cause this? I thought maybe the data might
be getting left in the caches, but I am waiting until the process exits
before calling m5 resetstats. The "bytesReadDRAM" is less than expected,
also, based on the number of loads in the instruction trace. I thought
perhaps this was because no-write allocate was being used, but the
discussion linked below suggests that default is to use write-allocate. I
can't find where this is configured in gem5, so I'm not able to check this
at the moment.
http://comments.gmane.org/gmane.comp.emulators.m5.users/12597
Any help is appreciated.
​-​
Patrick
-- IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ,
Registered in England & Wales, Company No: 2557590
ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1
9NJ, Registered in England & Wales, Company No: 2548782
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
Patrick
2015-04-22 14:49:24 UTC
Permalink
Prathap,

75 L2 misses, 26 L2 writebacks. 26 writebacks is what I would expect, given
the 1,664 bytesWritten reported by the memory controller. However, what
about the other 2,432 bytes in the array that were written? I dumped the
stats after the process exited, so I wouldn't expect any of the written
data at that point to be sitting in the caches.

Thanks for the response.
Patrick
Post by Prathap Kolakkampadath
Hello Patrick,
Can you check the number of last level cache misses as reported by
stats.text?
Prathap
Post by Patrick
I looked back at this, and I'm still not sure it's clear to me what is
going on. I decreased the size of the write queue to 2, and when running
the simulation described in my previous message (in which 512 64-bit stores
to unique addresses are issued), bytesWritten in one run was reported to be
only 1,664 bytes. With the write queue set to size 2, I would expect
bytesWritten to be at least 4096 - 128 = 3,968 bytes (the burstSize is 64
bytes).
Any additional help is appreciated.
Regards,
Patrick
Post by Patrick
Thanks, Andreas. This makes sense.
On Wed, Apr 15, 2015 at 5:26 PM, Andreas Hansson <
Post by Andreas Hansson
Hi Patrick,
When it comes to the stores you are looking at a rather small number
of operations, and my guess is that they are still in the DRAM write
queues. These queues are not drained at the moment once the writes fall
below the “low water mark”.
Andreas
Date: Wednesday, 15 April 2015 19:13
Subject: [gem5-users] bytesWritten < (8 * number of 64-bit stores to
unique addresses)
I am looking at stats.txt for the amount of data written to the DRAM
during the execution of a process in full system mode. I looked at the
execution trace, and there are at least 512 64-bit stores to unique
addresses. However, stats.txt reports only 2,304 "bytesWritten" to the
memory. It is a 4-channel memory configuration. "stats.txt" reports 1,152
"bytesWritten" on channel 0, 0 "bytesWritten" to channel 1, 0
"bytesWritten" to channel 2, and 1,152 "bytesWritten" to channel 3.
Does anyone know what would cause this? I thought maybe the data
might be getting left in the caches, but I am waiting until the process
exits before calling m5 resetstats. The "bytesReadDRAM" is less than
expected, also, based on the number of loads in the instruction trace.
I thought perhaps this was because no-write allocate was being used, but
the discussion linked below suggests that default is to use write-allocate.
I can't find where this is configured in gem5, so I'm not able to check
this at the moment.
http://comments.gmane.org/gmane.comp.emulators.m5.users/12597
Any help is appreciated.
​-​
Patrick
-- IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ,
Registered in England & Wales, Company No: 2557590
ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1
9NJ, Registered in England & Wales, Company No: 2548782
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________
gem5-users mailing list
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
Loading...