Join Log in
Score:0
John Goeller avatar Ubuntu

System with 1.25 TB memory OS only seeing 1.00 TB ram

gh flag
John Goeller

I have a System with 1.25 TB of ram the Mother board sees it , lshw see's it , but only 1.0 TB is useable, Is there a kernel Parameter I need to recognize more memory ?

It is an Asus Mother board RS700-E9 , with 2 Intel XEON Gold 6150 CPU's

[root@xtc ~]# lshw -C memory
  *-firmware
       description: BIOS
       vendor: American Megatrends Inc.
       physical id: 0
       version: 0403
       date: 07/04/2017
       size: 64KiB
       capacity: 16MiB
       capabilities: pci apm upgrade shadowing cdboot bootselect socketedrom edd int13floppy1200 int13floppy720 int13floppy2880 int5printscreen int9keyboard int14serial int17printer acpi usb biosbootspecification uefi
  *-memory
       description: System Memory
       physical id: 31
       slot: System board or motherboard
       size: 1280GiB                 <------------- memory in system 
       capabilities: ecc
       configuration: errordetection=ecc
     *-bank:0
          description: DIMM DDR4 Synchronous 2400 MHz (0.4 ns)
          product: HMAA8GL7MMR4N-UH
          vendor: Hynix
          physical id: 0
          serial: 32BF004A
          slot: DIMM_A1
          size: 64GiB
          width: 64 bits
          clock: 2400MHz (0.4ns)
.
.
.

but can only 1 TB

[root@xtc ~]# free -h
              total        used        free      shared  buff/cache   available
Mem:           1.0T         10G        994G         19M        346M        993G
Swap:          4.0G          0B        4.0G
[root@xtc ~]#

[root@xtc ~]# dmidecode -t memory
# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 3.0 present.

Handle 0x0031, DMI type 16, 23 bytes
Physical Memory Array
        Location: System Board Or Motherboard
        Use: System Memory
        Error Correction Type: Single-bit ECC
        Maximum Capacity: 768 GB     <---- thats Per CPU  
        Error Information Handle: Not Provided
        Number Of Devices: 6

Handle 0x0033, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x0031
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 64 GB
        Form Factor: DIMM
        Set: None
        Locator: DIMM_A1
        Bank Locator: NODE 1
        Type: DDR4
        Type Detail: Synchronous
        Speed: 2400 MT/s
        Manufacturer: Hynix
        Serial Number: 32BF004A
        Asset Tag: DIMM_A1_AssetTag
        Part Number: HMAA8GL7MMR4N-UH
        Rank: 4
        Configured Memory Speed: 2400 MT/s
        Minimum Voltage: 1.2 V
        Maximum Voltage: 1.2 V
        Configured Voltage: 1.2 V

Handle 0x0035, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x0031
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 64 GB
        Form Factor: DIMM
        Set: None
        Locator: DIMM_A2
        Bank Locator: NODE 1
        Type: DDR4
        Type Detail: Synchronous
        Speed: 2666 MT/s
        Manufacturer: Micron
        Serial Number: 353C140B
        Asset Tag: DIMM_A2_AssetTag
        Part Number: 72ASS8G72LZ-2G6J1
        Rank: 4
        Configured Memory Speed: 2400 MT/s
        Minimum Voltage: 1.2 V
        Maximum Voltage: 1.2 V
        Configured Voltage: 1.2 V

Handle 0x0037, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x0031
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 64 GB
        Form Factor: DIMM
        Set: None
        Locator: DIMM_B1
        Bank Locator: NODE 1
        Type: DDR4
        Type Detail: Synchronous
        Speed: 2400 MT/s
        Manufacturer: Hynix
        Serial Number: 32BF00A6
        Asset Tag: DIMM_B1_AssetTag
        Part Number: HMAA8GL7MMR4N-UH
        Rank: 4
        Configured Memory Speed: 2400 MT/s
        Minimum Voltage: 1.2 V
        Maximum Voltage: 1.2 V
        Configured Voltage: 1.2 V

Handle 0x0039, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x0031
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 64 GB
        Form Factor: DIMM
        Set: None
        Locator: DIMM_B2
        Bank Locator: NODE 1
        Type: DDR4
        Type Detail: Synchronous
        Speed: 2666 MT/s
        Manufacturer: Micron
        Serial Number: 353C2294
        Asset Tag: DIMM_B2_AssetTag
        Part Number: 72ASS8G72LZ-2G6J1
        Rank: 4
        Configured Memory Speed: 2400 MT/s
        Minimum Voltage: 1.2 V
        Maximum Voltage: 1.2 V
        Configured Voltage: 1.2 V

any ideas ????

****** adding the imxi -m output

 RAM: total: 1005.33 GiB used: 12.41 GiB (1.2%)
  Array-1: capacity: 768 GiB slots: 6 EC: Single-bit ECC
  Device-1: DIMM_A1 type: DDR4 size: 64 GiB speed: 2400 MT/s
  Device-2: DIMM_A2 type: DDR4 size: 64 GiB speed: spec: 2666 MT/s
  Device-3: DIMM_B1 type: DDR4 size: 64 GiB speed: 2400 MT/s
  Device-4: DIMM_B2 type: DDR4 size: 64 GiB speed: spec: 2666 MT/s
  Device-5: DIMM_C1 type: DDR4 size: 64 GiB speed: 2400 MT/s
  Device-6: DIMM_C2 type: no module installed
  Array-2: capacity: 768 GiB slots: 6 EC: Single-bit ECC
  Device-1: DIMM_D1 type: DDR4 size: 64 GiB speed: 2400 MT/s
  Device-2: DIMM_D2 type: DDR4 size: 64 GiB speed: spec: 2666 MT/s
  Device-3: DIMM_E1 type: DDR4 size: 64 GiB speed: 2400 MT/s
  Device-4: DIMM_E2 type: DDR4 size: 64 GiB speed: spec: 2666 MT/s
  Device-5: DIMM_F1 type: DDR4 size: 64 GiB speed: 2400 MT/s
  Device-6: DIMM_F2 type: no module installed
  Array-3: capacity: 768 GiB slots: 6 EC: Single-bit ECC
  Device-1: DIMM_G1 type: DDR4 size: 64 GiB speed: 2400 MT/s
  Device-2: DIMM_G2 type: DDR4 size: 64 GiB speed: spec: 2666 MT/s
  Device-3: DIMM_H1 type: DDR4 size: 64 GiB speed: 2400 MT/s
  Device-4: DIMM_H2 type: DDR4 size: 64 GiB speed: spec: 2666 MT/s
  Device-5: DIMM_J1 type: DDR4 size: 64 GiB speed: 2400 MT/s
  Device-6: DIMM_J2 type: no module installed
  Array-4: capacity: 768 GiB slots: 6 EC: Single-bit ECC
  Device-1: DIMM_K1 type: DDR4 size: 64 GiB speed: 2400 MT/s
  Device-2: DIMM_K2 type: DDR4 size: 64 GiB speed: spec: 2666 MT/s
  Device-3: DIMM_L1 type: DDR4 size: 64 GiB speed: 2400 MT/s
  Device-4: DIMM_L2 type: DDR4 size: 64 GiB speed: spec: 2666 MT/s
  Device-5: DIMM_M1 type: DDR4 size: 64 GiB speed: 2400 MT/s
  Device-6: DIMM_M2 type: no module installed

adding proc/meminfo

# cat /proc/meminfo
MemTotal:       1054167812 kB
MemFree:        1040596332 kB
MemAvailable:   1040131352 kB
Buffers:            1400 kB
Cached:          1305460 kB
SwapCached:            0 kB
Active:          1495260 kB
Inactive:         452332 kB
Active(anon):     643724 kB
Inactive(anon):    33940 kB
Active(file):     851536 kB
Inactive(file):   418392 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:      980955128 kB
SwapFree:       980955128 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:        638656 kB
Mapped:           166396 kB
Shmem:             36088 kB
Slab:             268216 kB
SReclaimable:      90680 kB
SUnreclaim:       177536 kB
KernelStack:       22864 kB
PageTables:        43764 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    1508039032 kB
Committed_AS:    5051712 kB
VmallocTotal:   34359738367 kB
VmallocUsed:     2269720 kB
VmallocChunk:   34081574908 kB
Percpu:            97280 kB
HardwareCorrupted:     0 kB
AnonHugePages:    221184 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      320044 kB
DirectMap2M:     9787392 kB
DirectMap1G:    1063256064 kB
106
0 + 15
ram
in flag
matigo
Welcome to AskUbuntu! Could you [edit] your question to include: (0) the version of Ubuntu you’re running (1) the terminal output of `sudo dmidecode -t memory` (2) the brand and model of your CPU and motherboard. With this, it may be possible to identify why only 1TB is accessible
0
Reply
in flag
matigo
Is the RAM seated in slots 0-19, or did you split it 0-9, then 12-21? This looks like one socket has only 512GB (as per DIMM layout), which will have the motherboard identify all of it, but the OS “balance” it by going with the minimum assigned to a given physical socket
1
Reply
Will avatar
id flag
Will
I think at least part of the discrepancy is that there is a difference between what free and lshw measure (GiB vs GB) - one binary, the other decimal. Have a look at the comments in https://askubuntu.com/questions/1130621/why-do-free-and-lshw-show-different-ram-sizes
0
Reply
Terrance avatar
id flag
Terrance
Try looking at it with `free --si -h` and then `free -h` to see how much difference is in reporting between GB vs GiB
1
Reply
us flag
thomasrutter
Even at terabyte level, the difference between TB and TiB is less than 10%, so it can't account for a 25% difference. At gigabyte level the difference is less than 7.5%.
1
Reply
Terrance avatar
id flag
Terrance
It could be a bug in the `free` command as well. What is the output of `sudo inxi -m`?
0
Reply
John Goeller avatar
gh flag
John Goeller
aded the imxi -m output , that is very interesting it shows all the memory dimm, but reports 1 TB of ram
0
Reply
ru flag
Thomas Ward
how many processors (physical CPUs, not cores, etc.) are on this board? There's a correlation between number of processors and usable ram on some boards, and i think anything over a TB needs to be at least dual-processors to work with more than 1TB of RAM on some boards
0
Reply
Matias N Goldberg avatar
vg flag
Matias N Goldberg
Instead of using `free -h` could you use simply `free` in bytes? There's a slight chance it's a rounding bug in how `free` reports memory and 1.2TiB ends up being incorrectly rounded down to 1.0TiB.
0
Reply
Matias N Goldberg avatar
vg flag
Matias N Goldberg
Can you post the output of `cat /proc/meminfo` ?
0
Reply
John Goeller avatar
gh flag
John Goeller
2 Intel Xeon 6150 Gold CPU
0
Reply
John Goeller avatar
gh flag
John Goeller
added Proc/meminfo above
0
Reply
John Goeller avatar
gh flag
John Goeller
I re-arranged the DImm's and either the system wouldn't boot , or got the same result
0
Reply
Terrance avatar
id flag
Terrance
Your configuration on the DIMMs is correct. https://dlcdnets.asus.com/pub/ASUS/server/RS700-E9-RS4/Manual/E19796_RS700-E9-RS4_RS12_UM_V7_WEB.pdf Maybe a BIOS update as your BIOS version is quite old from 2017 https://servers.asus.com/products/Servers/Rack-Servers/RS700-E9-RS12/#Resources
0
Reply
John Goeller avatar
gh flag
John Goeller
OK so I booted a USB stick contain CENTOS and gotthe same results. I also fully populatedthe machine to 1.5 TB , but still only have access to 1.tb . also took memory out ofthis machine and put it in another machine o to make it 1.25 TB and that Machine worked. that machne was a super micro , this one is an Asus . so It makes me believe it is some setting in the Bios
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.