Trace number 3262794

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.42 1209.01

General information on the benchmark

NameSAT03/handmade/anton/UNSAT/
SGI_30_60_24_40_2-log.shuffled-as.sat03-119.cnf
MD5SUMcf96a8eea16f98b3cec72117cedd0285
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables120
Number of clauses48342
Sum of the clauses size483180
Maximum clause length10
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 548
Number of clauses of size over 548294

Solver Data 

0.09/0.11	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262794-1304094214.cnf 
0.09/0.11	c Using SATTIMEOUT of 4800
0.09/0.11	c Using SATRAM of 15500
0.09/0.11	c Size of subset node = 32
0.09/0.11	c Variables = 120
0.09/0.11	c Clauses = 48342
0.09/0.11	c Literals = 483180
0.09/0.11	c Reduced size:
0.09/0.11	c Variables = 120
0.09/0.11	c Clauses = 48342
0.09/0.11	c Literals = 483180
0.09/0.11	c a2
0.79/0.86	c a1
2.39/2.43	c Independent sub-problems = 1
2.39/2.43	c Sub-problem with 120 variables and 48342 clauses
2.39/2.43	c using permutation method 4
2.39/2.43	c count = 1
2.39/2.43	c Ordering finished 2.390 seconds
2.39/2.43	c mapped 2.420 seconds
2.39/2.43	c permuted 2.420 seconds
2.39/2.43	c about to sort
2.39/2.45	c sorted
2.49/2.55	c clauses now 48342
2.49/2.55	c init finished 2.450 seconds
2.49/2.55	c tree built 2.540 seconds
2.49/2.55	c 
2.49/2.55	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
2.49/2.55	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
2.49/2.55	c ................................ | ..........                    |                                  2.5         0.0             
2.49/2.55	c ................................ | ..........                  1 |                                  2.5         0.4             
2.49/2.55	c ................................ | ..........                  2 |                                  2.5         0.8             
2.49/2.55	c ................................ | ..........                  3 |                                  2.5         1.2             
2.49/2.55	c ................................ | ..........                  4 |                                  2.5         1.6             
2.49/2.55	c ................................ | ..........                  5 |                                  2.5         2.0             
2.49/2.56	c ................................ | ..........               1000 | 116          1262                2.5       392.2         8248
96.01/26.01	c ................................ | ..........            1843000 | 468412       2491892            96.0     19187.9     25660335
191.98/50.12	c ................................ | ..........            3097000 | 788739       4139714           192.1     16125.2     37022721
276.94/71.49	c *............................... | ..........            4060946 | 1038689      5351211           277.3     14644.1     45647713
288.06/74.20	c *............................... | ..........            4179000 | 1069386      5505683           288.1     14506.9     46780260
384.08/98.31	c *............................... | ..........            5130000 | 1318666      6764860           384.1     13354.5     54333627
479.63/122.39	c *............................... | *.........            6009000 | 1547534      7936630           480.0     12518.5     58237845
575.95/146.52	c *............................... | *.........            6860000 | 1770799      9051716           576.0     11909.1     64167813
671.88/170.65	c *............................... | *.........            7580000 | 1958532      10078017          672.1     11278.4     68828461
767.84/194.76	c *............................... | *.........            8342000 | 2155800      11078693          768.1     10860.8     74280632
863.73/218.90	c *............................... | *.........            9082000 | 2350138      12062475          864.1     10510.0     79188850
960.04/243.02	c *............................... | **........            9775000 | 2531878      12993341          960.1     10180.9     82809507
1055.97/267.13	c *............................... | **........           10497000 | 2722210      13938774         1056.1      9939.3     84263414
1151.86/291.24	c *............................... | **........           11167000 | 2896916      14868780         1152.0      9693.4     86411587
1247.97/315.41	c *............................... | **........           11829000 | 3070905      15797076         1248.0      9478.1     90986785
1343.87/339.54	c *............................... | **........           12455000 | 3236189      16697532         1344.0      9266.8     90986785
1439.85/363.63	c *............................... | ***.......           13066000 | 3399280      17545188         1440.0      9073.5     91403339
1535.92/387.73	c *............................... | ***.......           13665000 | 3558526      18420615         1536.1      8896.1     93703045
1631.91/411.82	c *............................... | ***.......           14230000 | 3706992      19262254         1632.0      8719.3     97947608
1727.71/435.99	c *............................... | ***.......           14817000 | 3859099      20093503         1728.1      8574.2    101407863
1823.89/460.13	c *............................... | ***.......           15423000 | 4016483      20917114         1824.0      8455.5    103384371
1911.61/482.27	c **.............................. | ***.......           15952836 | 4154171      21678547         1911.9      8343.9    104659517
1911.61/482.27	c **.............................. | ***.......           15952837 | 4154171      21678550         1911.9      8343.9    104659547
1919.96/484.32	c **.............................. | ****......           16013000 | 4169650      21753866         1920.0      8339.9    104950950
2015.93/508.44	c **.............................. | ****......           16696000 | 4348871      22578406         2016.1      8281.3    106613126
2111.63/532.59	c **.............................. | ****......           17384000 | 4527559      23424379         2112.0      8230.9    107000958
2207.89/556.75	c **.............................. | ****......           18048000 | 4703154      24261871         2208.1      8173.6    108183621
2303.81/580.86	c **.............................. | ****......           18748000 | 4884353      25083908         2304.1      8136.9    109822345
2400.09/605.01	c **.............................. | *****.....           19350000 | 5043138      25864710         2400.1      8062.1    110766063
2496.01/629.14	c **.............................. | *****.....           19915000 | 5190657      26637177         2496.2      7978.2    112069278
2592.18/653.30	c **.............................. | *****.....           20519000 | 5347992      27422211         2592.2      7915.7    112414261
2687.70/677.38	c **.............................. | *****.....           21091000 | 5497733      28183867         2688.0      7846.3    113557974
2783.83/701.55	c **.............................. | *****.....           21613000 | 5634871      28914839         2784.0      7763.2    114586737
2880.00/725.70	c **.............................. | ******....           22228000 | 5795312      29699533         2880.0      7718.0    115851123
2975.82/749.85	c **.............................. | ******....           22855000 | 5959051      30464539         2976.0      7679.7    116926089
3072.10/774.00	c **.............................. | ******....           23422000 | 6106444      31228947         3072.1      7624.1    118214091
3168.04/798.11	c **.............................. | ******....           24019000 | 6263238      31973679         3168.1      7581.5    118988341
3263.95/822.22	c **.............................. | ******....           24633000 | 6424112      32747077         3264.1      7546.8    119141819
3359.98/846.40	c **.............................. | *******...           25193000 | 6571397      33497458         3360.0      7497.9    119798859
3455.79/870.57	c **.............................. | *******...           25736000 | 6716031      34230452         3456.1      7446.6    120594459
3552.11/894.73	c **.............................. | *******...           26231000 | 6844607      34960211         3552.2      7384.4    122800755
3648.00/918.82	c **.............................. | *******...           26794000 | 6992351      35724406         3648.1      7344.7    122800755
3743.77/942.99	c **.............................. | *******...           27384000 | 7147435      36483608         3744.1      7313.8    123062181
3839.90/967.15	c **.............................. | ********..           27974000 | 7304834      37237697         3840.1      7284.7    123951679
3934.27/990.88	c ***............................. | ********..           28526665 | 7450844      37970345         3934.6      7250.2    124633210
3935.82/991.27	c ***............................. | ********..           28535000 | 7452942      37981271         3936.1      7249.6    124643590
4031.80/1015.36	c ***............................. | ********..           29092000 | 7596921      38742710         4032.0      7215.2    126076029
4127.92/1039.53	c ***............................. | ********..           29603000 | 7728437      39453276         4128.0      7171.2    127558887
4223.97/1063.74	c ***............................. | ********..           30141000 | 7868239      40189985         4224.1      7135.4    129425337
4319.93/1087.94	c ***............................. | *********.           30666000 | 8005522      40925974         4320.1      7098.4    130802478
4415.84/1112.08	c ***............................. | *********.           31162000 | 8136540      41639161         4416.1      7056.4    131221630
4511.74/1136.29	c ***............................. | *********.           31656000 | 8266458      42311151         4512.1      7015.8    131947108
4607.65/1160.39	c ***............................. | *********.           32115000 | 8386056      43001002         4608.0      6969.4    132763554
4703.87/1184.56	c ***............................. | *********.           32594000 | 8510974      43678616         4704.1      6928.8    133288393
4799.87/1208.77	c CPU limit (4800) reached
4799.87/1208.77	c Time = 4800.17 seconds
4799.87/1208.77	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3262794-1304094214/watcher-3262794-1304094214 -o /tmp/evaluation-result-3262794-1304094214/solver-3262794-1304094214 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262794-1304094214.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.02 8.08 8.00 1/179 10323
/proc/meminfo: memFree=25016476/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=8972 CPUtime=0 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) R 10321 10323 9884 0 -1 4202496 328 0 0 0 0 0 0 0 25 0 1 0 521019652 9187328 270 33554432000 4194304 4338100 140736196122272 18446744073709551615 246516834048 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/10323/statm: 2243 270 147 36 0 148 0

[startup+0.0237259 s]
/proc/loadavg: 8.02 8.08 8.00 1/179 10323
/proc/meminfo: memFree=25016476/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=10292 CPUtime=0.02 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) R 10321 10323 9884 0 -1 4202496 645 0 0 0 2 0 0 0 25 0 1 0 521019652 10539008 587 33554432000 4194304 4338100 140736196122272 18446744073709551615 246516495045 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/10323/statm: 2573 588 147 36 0 478 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 10292

[startup+0.100693 s]
/proc/loadavg: 8.02 8.08 8.00 1/179 10323
/proc/meminfo: memFree=25016476/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=15924 CPUtime=0.09 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) R 10321 10323 9884 0 -1 4202496 2024 0 0 0 9 0 0 0 25 0 1 0 521019652 16306176 1966 33554432000 4194304 4338100 140736196122272 18446744073709551615 246516495045 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/10323/statm: 3981 1967 148 36 0 1886 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 15924

[startup+0.300693 s]
/proc/loadavg: 8.02 8.08 8.00 1/179 10323
/proc/meminfo: memFree=25016476/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=76872 CPUtime=0.29 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) R 10321 10323 9884 0 -1 4202496 9401 0 0 0 29 0 0 0 25 0 1 0 521019652 78716928 8966 33554432000 4194304 4338100 140736196122272 18446744073709551615 246516229841 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/10323/statm: 19218 8966 298 36 0 17123 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 76872

[startup+0.700614 s]
/proc/loadavg: 8.02 8.08 8.00 1/179 10323
/proc/meminfo: memFree=25016476/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=76872 CPUtime=0.69 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) R 10321 10323 9884 0 -1 4202496 9401 0 0 0 69 0 0 0 25 0 1 0 521019652 78716928 8966 33554432000 4194304 4338100 140736196122272 18446744073709551615 246516229336 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/10323/statm: 19218 8966 298 36 0 17123 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 76872

[startup+1.5004 s]
/proc/loadavg: 7.54 7.97 7.97 7/186 10355
/proc/meminfo: memFree=24955636/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=21932 CPUtime=1.49 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) R 10321 10323 9884 0 -1 4202496 10567 0 0 0 149 0 0 0 25 0 1 0 521019652 22458368 3471 33554432000 4194304 4338100 140736196122272 18446744073709551615 4245192 0 0 4096 0 0 0 0 17 0 0 0 0
/proc/10323/statm: 5483 3471 155 36 0 3388 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 21932

[startup+3.10108 s]
/proc/loadavg: 7.54 7.97 7.97 7/186 10355
/proc/meminfo: memFree=24922280/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=319800 CPUtime=4.71 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) S 10321 10323 9884 0 -1 4202496 24817 0 0 0 468 3 0 0 25 0 5 0 521019652 327475200 17721 33554432000 4194304 4338100 140736196122272 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10323/statm: 79950 17721 176 36 0 77855 0
Current children cumulated CPU time (s) 4.71
Current children cumulated vsize (KiB) 319800

[startup+6.3005 s]
/proc/loadavg: 7.54 7.97 7.97 9/190 10359
/proc/meminfo: memFree=24658640/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=499548 CPUtime=17.48 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) S 10321 10323 9884 0 -1 4202496 73306 0 0 0 1735 13 0 0 25 0 5 0 521019652 511537152 66210 33554432000 4194304 4338100 140736196122272 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10323/statm: 124887 66210 176 36 0 122792 0
[pid=10323/tid=10356] ppid=10321 vsize=499548 CPUtime=3.72 cores=0,2,4,6
/proc/10323/task/10356/stat : 10356 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 14989 0 0 0 370 2 0 0 25 0 5 0 521019908 511537152 66210 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10323/tid=10357] ppid=10321 vsize=499548 CPUtime=3.7 cores=0,2,4,6
/proc/10323/task/10357/stat : 10357 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 16134 0 0 0 367 3 0 0 25 0 5 0 521019908 511537152 66211 33554432000 4194304 4338100 140736196122272 18446744073709551615 4239500 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10323/tid=10358] ppid=10321 vsize=499548 CPUtime=3.73 cores=0,2,4,6
/proc/10323/task/10358/stat : 10358 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 15275 0 0 0 370 3 0 0 25 0 5 0 521019908 511537152 66212 33554432000 4194304 4338100 140736196122272 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10323/tid=10359] ppid=10321 vsize=499548 CPUtime=3.73 cores=0,2,4,6
/proc/10323/task/10359/stat : 10359 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 15287 0 0 0 370 3 0 0 25 0 5 0 521019908 511537152 66212 33554432000 4194304 4338100 140736196122272 18446744073709551615 4256288 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 17.48
Current children cumulated vsize (KiB) 499548

[startup+12.7003 s]
/proc/loadavg: 7.76 8.01 7.98 9/190 10359
/proc/meminfo: memFree=24283400/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=745508 CPUtime=42.99 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) S 10321 10323 9884 0 -1 4202496 138706 0 0 0 4272 27 0 0 25 0 5 0 521019652 763400192 131610 33554432000 4194304 4338100 140736196122272 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10323/statm: 186377 131610 176 36 0 184282 0
[pid=10323/tid=10356] ppid=10321 vsize=745508 CPUtime=10.11 cores=0,2,4,6
/proc/10323/task/10356/stat : 10356 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 31353 0 0 0 1004 7 0 0 25 0 5 0 521019908 763400192 131610 33554432000 4194304 4338100 140736196122272 18446744073709551615 4263461 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10323/tid=10357] ppid=10321 vsize=745508 CPUtime=10.05 cores=0,2,4,6
/proc/10323/task/10357/stat : 10357 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 34150 0 0 0 999 6 0 0 25 0 5 0 521019908 763400192 131610 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254384 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10323/tid=10358] ppid=10321 vsize=745508 CPUtime=10.11 cores=0,2,4,6
/proc/10323/task/10358/stat : 10358 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 31618 0 0 0 1005 6 0 0 21 0 5 0 521019908 763400192 131610 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10323/tid=10359] ppid=10321 vsize=745508 CPUtime=10.12 cores=0,2,4,6
/proc/10323/task/10359/stat : 10359 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 29963 0 0 0 1007 5 0 0 21 0 5 0 521019908 763400192 131611 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 42.99
Current children cumulated vsize (KiB) 745508

[startup+25.501 s]
/proc/loadavg: 7.79 8.01 7.98 9/190 10360
/proc/meminfo: memFree=23765084/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=1088712 CPUtime=94.02 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) S 10321 10323 9884 0 -1 4202496 229581 0 0 0 9351 51 0 0 25 0 5 0 521019652 1114841088 222485 33554432000 4194304 4338100 140736196122272 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10323/statm: 272178 222485 176 36 0 270083 0
[pid=10323/tid=10356] ppid=10321 vsize=1088712 CPUtime=22.89 cores=0,2,4,6
/proc/10323/task/10356/stat : 10356 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 55412 0 0 0 2276 13 0 0 25 0 5 0 521019908 1114841088 222485 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10323/tid=10357] ppid=10321 vsize=1088712 CPUtime=22.75 cores=0,2,4,6
/proc/10323/task/10357/stat : 10357 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 57232 0 0 0 2262 13 0 0 20 0 5 0 521019908 1114841088 222485 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10323/tid=10358] ppid=10321 vsize=1088712 CPUtime=22.89 cores=0,2,4,6
/proc/10323/task/10358/stat : 10358 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 52867 0 0 0 2277 12 0 0 25 0 5 0 521019908 1114841088 222485 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10323/tid=10359] ppid=10321 vsize=1088712 CPUtime=22.91 cores=0,2,4,6
/proc/10323/task/10359/stat : 10359 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 52448 0 0 0 2280 11 0 0 25 0 5 0 521019908 1114841088 222485 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 94.02
Current children cumulated vsize (KiB) 1088712

[startup+51.1003 s]
/proc/loadavg: 7.86 8.00 7.98 9/190 10360
/proc/meminfo: memFree=23131852/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=1500308 CPUtime=195.97 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) S 10321 10323 9884 0 -1 4202496 341293 0 0 0 19509 88 0 0 25 0 5 0 521019652 1536315392 334197 33554432000 4194304 4338100 140736196122272 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10323/statm: 375077 334197 176 36 0 372982 0
[pid=10323/tid=10356] ppid=10321 vsize=1500308 CPUtime=48.31 cores=0,2,4,6
/proc/10323/task/10356/stat : 10356 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 86684 0 0 0 4807 24 0 0 25 0 5 0 521019908 1536315392 334197 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10323/tid=10357] ppid=10321 vsize=1500308 CPUtime=48.21 cores=0,2,4,6
/proc/10323/task/10357/stat : 10357 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 83303 0 0 0 4798 23 0 0 25 0 5 0 521019908 1536315392 334197 33554432000 4194304 4338100 140736196122272 18446744073709551615 246516229172 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10323/tid=10358] ppid=10321 vsize=1500308 CPUtime=48.43 cores=0,2,4,6
/proc/10323/task/10358/stat : 10358 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 77486 0 0 0 4824 19 0 0 25 0 5 0 521019908 1536315392 334197 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10323/tid=10359] ppid=10321 vsize=1500308 CPUtime=48.42 cores=0,2,4,6
/proc/10323/task/10359/stat : 10359 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 82198 0 0 0 4823 19 0 0 25 0 5 0 521019908 1536315392 334197 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 195.97
Current children cumulated vsize (KiB) 1500308

[startup+102.307 s]
/proc/loadavg: 7.94 8.00 7.98 9/190 10362
/proc/meminfo: memFree=22139208/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=2056408 CPUtime=400.02 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) S 10321 10323 9884 0 -1 4202496 496839 0 0 0 39867 135 0 0 25 0 5 0 521019652 2105761792 489743 33554432000 4194304 4338100 140736196122272 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10323/statm: 514102 489743 176 36 0 512007 0
[pid=10323/tid=10356] ppid=10321 vsize=2056408 CPUtime=99.16 cores=0,2,4,6
/proc/10323/task/10356/stat : 10356 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 135825 0 0 0 9877 39 0 0 25 0 5 0 521019908 2105761792 489743 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10323/tid=10357] ppid=10321 vsize=2056408 CPUtime=99.36 cores=0,2,4,6
/proc/10323/task/10357/stat : 10357 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 119023 0 0 0 9900 36 0 0 25 0 5 0 521019908 2105761792 489744 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10323/tid=10358] ppid=10321 vsize=2056408 CPUtime=99.51 cores=0,2,4,6
/proc/10323/task/10358/stat : 10358 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 113940 0 0 0 9924 27 0 0 25 0 5 0 521019908 2105761792 489744 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10323/tid=10359] ppid=10321 vsize=2056408 CPUtime=99.41 cores=0,2,4,6
/proc/10323/task/10359/stat : 10359 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 116429 0 0 0 9910 31 0 0 25 0 5 0 521019908 2105761792 489744 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 400.02
Current children cumulated vsize (KiB) 2056408

[startup+162.301 s]
/proc/loadavg: 8.19 8.07 8.01 9/190 10363
/proc/meminfo: memFree=21401640/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=2661836 CPUtime=638.81 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) S 10321 10323 9884 0 -1 4202496 615668 0 0 0 63700 181 0 0 25 0 5 0 521019652 2725720064 608572 33554432000 4194304 4338100 140736196122272 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10323/statm: 665459 608572 176 36 0 663364 0
[pid=10323/tid=10356] ppid=10321 vsize=2661836 CPUtime=158.73 cores=0,2,4,6
/proc/10323/task/10356/stat : 10356 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 179248 0 0 0 15817 56 0 0 25 0 5 0 521019908 2725720064 608572 33554432000 4194304 4338100 140736196122272 18446744073709551615 246516229243 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10323/tid=10357] ppid=10321 vsize=2661836 CPUtime=159.19 cores=0,2,4,6
/proc/10323/task/10357/stat : 10357 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 144847 0 0 0 15873 46 0 0 25 0 5 0 521019908 2725720064 608572 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10323/tid=10358] ppid=10321 vsize=2661836 CPUtime=159.19 cores=0,2,4,6
/proc/10323/task/10358/stat : 10358 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 139992 0 0 0 15881 38 0 0 25 0 5 0 521019908 2725720064 608572 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10323/tid=10359] ppid=10321 vsize=2661836 CPUtime=159.12 cores=0,2,4,6
/proc/10323/task/10359/stat : 10359 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 139959 0 0 0 15873 39 0 0 25 0 5 0 521019908 2725720064 608572 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 638.81
Current children cumulated vsize (KiB) 2661836

[startup+222.301 s]
/proc/loadavg: 8.20 8.09 8.01 10/190 10365
/proc/meminfo: memFree=20621496/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=3087748 CPUtime=877.68 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) S 10321 10323 9884 0 -1 4202496 736352 0 0 0 87546 222 0 0 25 0 5 0 521019652 3161853952 729256 33554432000 4194304 4338100 140736196122272 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10323/statm: 771937 729256 176 36 0 769842 0
[pid=10323/tid=10356] ppid=10321 vsize=3087748 CPUtime=218.19 cores=0,2,4,6
/proc/10323/task/10356/stat : 10356 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 222980 0 0 0 21749 70 0 0 25 0 5 0 521019908 3161853952 729256 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10323/tid=10357] ppid=10321 vsize=3087748 CPUtime=219.01 cores=0,2,4,6
/proc/10323/task/10357/stat : 10357 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 169688 0 0 0 21845 56 0 0 25 0 5 0 521019908 3161853952 729256 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10323/tid=10358] ppid=10321 vsize=3087748 CPUtime=219.02 cores=0,2,4,6
/proc/10323/task/10358/stat : 10358 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 168782 0 0 0 21854 48 0 0 25 0 5 0 521019908 3161853952 729256 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10323/tid=10359] ppid=10321 vsize=3087748 CPUtime=218.87 cores=0,2,4,6
/proc/10323/task/10359/stat : 10359 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 163280 0 0 0 21842 45 0 0 25 0 5 0 521019908 3161853952 729256 33554432000 4194304 4338100 140736196122272 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 877.68
Current children cumulated vsize (KiB) 3087748

################
# More data... #
################

/proc/10323/task/10359/stat : 10359 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 245806 0 0 0 69585 85 0 0 25 0 5 0 521019908 4881702912 1141666 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2787.02
Current children cumulated vsize (KiB) 4767288

[startup+762.306 s]
/proc/loadavg: 8.08 8.09 8.02 9/190 10379
/proc/meminfo: memFree=17191788/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=4868864 CPUtime=3025.43 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) S 10321 10323 9884 0 -1 4202496 1186583 0 0 0 302077 466 0 0 25 0 5 0 521019652 4985716736 1179487 33554432000 4194304 4338100 140736196122272 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10323/statm: 1217216 1179487 176 36 0 1215121 0
[pid=10323/tid=10356] ppid=10321 vsize=4868864 CPUtime=753.52 cores=0,2,4,6
/proc/10323/task/10356/stat : 10356 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 441338 0 0 0 75169 183 0 0 25 0 5 0 521019908 4985716736 1179487 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10323/tid=10357] ppid=10321 vsize=4868864 CPUtime=756.56 cores=0,2,4,6
/proc/10323/task/10357/stat : 10357 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 243106 0 0 0 75557 99 0 0 25 0 5 0 521019908 4985716736 1179487 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10323/tid=10358] ppid=10321 vsize=4868864 CPUtime=756.26 cores=0,2,4,6
/proc/10323/task/10358/stat : 10358 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 238674 0 0 0 75536 90 0 0 25 0 5 0 521019908 4985716736 1179487 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10323/tid=10359] ppid=10321 vsize=4868864 CPUtime=756.48 cores=0,2,4,6
/proc/10323/task/10359/stat : 10359 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 251843 0 0 0 75558 90 0 0 25 0 5 0 521019908 4985716736 1179487 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3025.43
Current children cumulated vsize (KiB) 4868864

[startup+822.3 s]
/proc/loadavg: 8.12 8.10 8.03 10/190 10381
/proc/meminfo: memFree=16927516/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=4946008 CPUtime=3264.35 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) S 10321 10323 9884 0 -1 4202496 1215499 0 0 0 325947 488 0 0 25 0 5 0 521019652 5064712192 1208403 33554432000 4194304 4338100 140736196122272 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10323/statm: 1236502 1208403 176 36 0 1234407 0
[pid=10323/tid=10356] ppid=10321 vsize=4946008 CPUtime=813.12 cores=0,2,4,6
/proc/10323/task/10356/stat : 10356 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 457346 0 0 0 81118 194 0 0 25 0 5 0 521019908 5064712192 1208403 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10323/tid=10357] ppid=10321 vsize=4946008 CPUtime=816.42 cores=0,2,4,6
/proc/10323/task/10357/stat : 10357 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 246878 0 0 0 81538 104 0 0 25 0 5 0 521019908 5064712192 1208403 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10323/tid=10358] ppid=10321 vsize=4946008 CPUtime=815.89 cores=0,2,4,6
/proc/10323/task/10358/stat : 10358 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 242909 0 0 0 81496 93 0 0 25 0 5 0 521019908 5064712192 1208403 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254352 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10323/tid=10359] ppid=10321 vsize=4946008 CPUtime=816.34 cores=0,2,4,6
/proc/10323/task/10359/stat : 10359 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 256744 0 0 0 81539 95 0 0 25 0 5 0 521019908 5064712192 1208403 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3264.35
Current children cumulated vsize (KiB) 4946008

[startup+882.3 s]
/proc/loadavg: 8.13 8.11 8.03 10/190 10383
/proc/meminfo: memFree=16752988/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=5216700 CPUtime=3502.78 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) S 10321 10323 9884 0 -1 4202496 1245097 0 0 0 349770 508 0 0 25 0 5 0 521019652 5341900800 1238001 33554432000 4194304 4338100 140736196122272 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10323/statm: 1304175 1238001 176 36 0 1302080 0
[pid=10323/tid=10356] ppid=10321 vsize=5216700 CPUtime=872.48 cores=0,2,4,6
/proc/10323/task/10356/stat : 10356 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 474131 0 0 0 87044 204 0 0 25 0 5 0 521019908 5341900800 1238001 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10323/tid=10357] ppid=10321 vsize=5216700 CPUtime=876.14 cores=0,2,4,6
/proc/10323/task/10357/stat : 10357 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 250482 0 0 0 87506 108 0 0 25 0 5 0 521019908 5341900800 1238001 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10323/tid=10358] ppid=10321 vsize=5216700 CPUtime=875.48 cores=0,2,4,6
/proc/10323/task/10358/stat : 10358 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 246441 0 0 0 87453 95 0 0 25 0 5 0 521019908 5341900800 1238001 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10323/tid=10359] ppid=10321 vsize=5216700 CPUtime=876.09 cores=0,2,4,6
/proc/10323/task/10359/stat : 10359 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 262421 0 0 0 87510 99 0 0 25 0 5 0 521019908 5341900800 1238001 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254451 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3502.78
Current children cumulated vsize (KiB) 5216700

[startup+942.3 s]
/proc/loadavg: 8.16 8.12 8.04 9/190 10384
/proc/meminfo: memFree=16560504/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=5314656 CPUtime=3741.38 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) S 10321 10323 9884 0 -1 4202496 1276514 0 0 0 373607 531 0 0 25 0 5 0 521019652 5442207744 1269418 33554432000 4194304 4338100 140736196122272 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10323/statm: 1328664 1269418 176 36 0 1326569 0
[pid=10323/tid=10356] ppid=10321 vsize=5314656 CPUtime=931.91 cores=0,2,4,6
/proc/10323/task/10356/stat : 10356 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 483700 0 0 0 92979 212 0 0 25 0 5 0 521019908 5442207744 1269418 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10323/tid=10357] ppid=10321 vsize=5314656 CPUtime=935.82 cores=0,2,4,6
/proc/10323/task/10357/stat : 10357 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 257836 0 0 0 93470 112 0 0 25 0 5 0 521019908 5442207744 1269418 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254285 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10323/tid=10358] ppid=10321 vsize=5314656 CPUtime=935.17 cores=0,2,4,6
/proc/10323/task/10358/stat : 10358 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 251630 0 0 0 93417 100 0 0 25 0 5 0 521019908 5442207744 1269418 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254439 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10323/tid=10359] ppid=10321 vsize=5314656 CPUtime=935.88 cores=0,2,4,6
/proc/10323/task/10359/stat : 10359 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 271726 0 0 0 93484 104 0 0 25 0 5 0 521019908 5442207744 1269418 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3741.38
Current children cumulated vsize (KiB) 5314656

[startup+1002.3 s]
/proc/loadavg: 8.19 8.13 8.04 10/190 10386
/proc/meminfo: memFree=16359948/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=5388212 CPUtime=3980.03 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) S 10321 10323 9884 0 -1 4202496 1306026 0 0 0 397450 553 0 0 25 0 5 0 521019652 5517529088 1298930 33554432000 4194304 4338100 140736196122272 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10323/statm: 1347053 1298930 176 36 0 1344958 0
[pid=10323/tid=10356] ppid=10321 vsize=5388212 CPUtime=991.54 cores=0,2,4,6
/proc/10323/task/10356/stat : 10356 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 501855 0 0 0 98931 223 0 0 25 0 5 0 521019908 5517529088 1298930 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10323/tid=10357] ppid=10321 vsize=5388212 CPUtime=995.45 cores=0,2,4,6
/proc/10323/task/10357/stat : 10357 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 261226 0 0 0 99428 117 0 0 25 0 5 0 521019908 5517529088 1298930 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10323/tid=10358] ppid=10321 vsize=5388212 CPUtime=994.85 cores=0,2,4,6
/proc/10323/task/10358/stat : 10358 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 255671 0 0 0 99384 101 0 0 25 0 5 0 521019908 5517529088 1298930 33554432000 4194304 4338100 140736196122272 18446744073709551615 4230450 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10323/tid=10359] ppid=10321 vsize=5388212 CPUtime=995.6 cores=0,2,4,6
/proc/10323/task/10359/stat : 10359 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 275652 0 0 0 99451 109 0 0 25 0 5 0 521019908 5517529088 1298930 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3980.03
Current children cumulated vsize (KiB) 5388212

[startup+1062.3 s]
/proc/loadavg: 8.26 8.16 8.06 10/190 10387
/proc/meminfo: memFree=16053132/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=5540100 CPUtime=4218.41 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) S 10321 10323 9884 0 -1 4202496 1354649 0 0 0 421262 579 0 0 25 0 5 0 521019652 5673062400 1347553 33554432000 4194304 4338100 140736196122272 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10323/statm: 1385025 1347553 176 36 0 1382930 0
[pid=10323/tid=10356] ppid=10321 vsize=5540100 CPUtime=1051.02 cores=0,2,4,6
/proc/10323/task/10356/stat : 10356 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 526230 0 0 0 104867 235 0 0 25 0 5 0 521019908 5673062400 1347553 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10323/tid=10357] ppid=10321 vsize=5540100 CPUtime=1055.13 cores=0,2,4,6
/proc/10323/task/10357/stat : 10357 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 264718 0 0 0 105391 122 0 0 25 0 5 0 521019908 5673062400 1347553 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254462 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10323/tid=10358] ppid=10321 vsize=5540100 CPUtime=1054.44 cores=0,2,4,6
/proc/10323/task/10358/stat : 10358 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 259216 0 0 0 105340 104 0 0 25 0 5 0 521019908 5673062400 1347553 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10323/tid=10359] ppid=10321 vsize=5540100 CPUtime=1055.24 cores=0,2,4,6
/proc/10323/task/10359/stat : 10359 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 292863 0 0 0 105408 116 0 0 25 0 5 0 521019908 5673062400 1347553 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4218.41
Current children cumulated vsize (KiB) 5540100

[startup+1122.3 s]
/proc/loadavg: 8.23 8.16 8.06 11/190 10389
/proc/meminfo: memFree=15858512/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=5622512 CPUtime=4456.63 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) S 10321 10323 9884 0 -1 4202496 1387128 0 0 0 445063 600 0 0 25 0 5 0 521019652 5757452288 1380032 33554432000 4194304 4338100 140736196122272 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10323/statm: 1405628 1380032 176 36 0 1403533 0
[pid=10323/tid=10356] ppid=10321 vsize=5622512 CPUtime=1110.5 cores=0,2,4,6
/proc/10323/task/10356/stat : 10356 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 541406 0 0 0 110806 244 0 0 25 0 5 0 521019908 5757452288 1380032 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10323/tid=10357] ppid=10321 vsize=5622512 CPUtime=1114.61 cores=0,2,4,6
/proc/10323/task/10357/stat : 10357 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 270131 0 0 0 111333 128 0 0 25 0 5 0 521019908 5757452288 1380032 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10323/tid=10358] ppid=10321 vsize=5622512 CPUtime=1113.94 cores=0,2,4,6
/proc/10323/task/10358/stat : 10358 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 262611 0 0 0 111289 105 0 0 25 0 5 0 521019908 5757452288 1380032 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10323/tid=10359] ppid=10321 vsize=5622512 CPUtime=1115 cores=0,2,4,6
/proc/10323/task/10359/stat : 10359 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 301358 0 0 0 111379 121 0 0 25 0 5 0 521019908 5757452288 1380032 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4456.63
Current children cumulated vsize (KiB) 5622512

[startup+1182.3 s]
/proc/loadavg: 8.25 8.18 8.07 9/190 10391
/proc/meminfo: memFree=15649776/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=5929980 CPUtime=4695.1 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) S 10321 10323 9884 0 -1 4202496 1416409 0 0 0 468887 623 0 0 25 0 5 0 521019652 6072299520 1409313 33554432000 4194304 4338100 140736196122272 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10323/statm: 1482495 1409313 176 36 0 1480400 0
[pid=10323/tid=10356] ppid=10321 vsize=5929980 CPUtime=1169.73 cores=0,2,4,6
/proc/10323/task/10356/stat : 10356 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 554509 0 0 0 116721 252 0 0 25 0 5 0 521019908 6072299520 1409313 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10323/tid=10357] ppid=10321 vsize=5929980 CPUtime=1174.34 cores=0,2,4,6
/proc/10323/task/10357/stat : 10357 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 275703 0 0 0 117300 134 0 0 25 0 5 0 521019908 6072299520 1409313 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10323/tid=10358] ppid=10321 vsize=5929980 CPUtime=1173.64 cores=0,2,4,6
/proc/10323/task/10358/stat : 10358 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 266005 0 0 0 117256 108 0 0 25 0 5 0 521019908 6072299520 1409313 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10323/tid=10359] ppid=10321 vsize=5929980 CPUtime=1174.8 cores=0,2,4,6
/proc/10323/task/10359/stat : 10359 (ttsth-5-4) R 10321 10323 9884 0 -1 4202560 308570 0 0 0 117353 127 0 0 25 0 5 0 521019908 6072299520 1409313 33554432000 4194304 4338100 140736196122272 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4695.1
Current children cumulated vsize (KiB) 5929980



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1208.8 s]
/proc/loadavg: 7.84 8.09 8.05 2/177 10409
/proc/meminfo: memFree=20039476/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=0 CPUtime=4800.21 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) Z 10321 10323 9884 0 -1 4203548 1422721 0 0 0 479381 640 0 0 18 0 2 0 521019652 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10323/statm: 0 0 0 0 0 0 0
[pid=10323/tid=10358] ppid=10321 vsize=0 CPUtime=1200.05 cores=0,2,4,6
/proc/10323/task/10358/stat : 10358 (ttsth-5-4) R 10321 10323 9884 0 -1 4203588 267465 0 0 0 119892 113 0 0 25 0 2 0 521019908 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.21
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1209 s]
/proc/loadavg: 7.84 8.09 8.05 2/177 10409
/proc/meminfo: memFree=20039476/32951124 swapFree=67111352/67111528
[pid=10323] ppid=10321 vsize=0 CPUtime=4800.39 cores=0,2,4,6
/proc/10323/stat : 10323 (ttsth-5-4) Z 10321 10323 9884 0 -1 4203548 1422721 0 0 0 479381 658 0 0 18 0 2 0 521019652 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/10323/statm: 0 0 0 0 0 0 0
[pid=10323/tid=10358] ppid=10321 vsize=0 CPUtime=1200.24 cores=0,2,4,6
/proc/10323/task/10358/stat : 10358 (ttsth-5-4) R 10321 10323 9884 0 -1 4203588 267465 0 0 0 119892 132 0 0 25 0 2 0 521019908 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.39
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1209.01
CPU time (s): 4800.42
CPU user time (s): 4793.82
CPU system time (s): 6.602
CPU usage (%): 397.052
Max. virtual memory (cumulated for all children) (KiB): 5935788

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4793.82
system time used= 6.602
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1422721
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 23737
involuntary context switches= 76920

runsolver used 2.10468 second user time and 4.50031 second system time

The end

Launcher Data 

Begin job on node134 at 2011-04-29 18:23:35
IDJOB=3262794
IDBENCH=83894
IDSOLVER=1654
FILE ID=node134/3262794-1304094214
RUNJOBID= node134-1304093010-10231
PBS_JOBID= 13170571
Free space on /tmp= 71584 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT03/handmade/anton/UNSAT/SGI_30_60_24_40_2-log.shuffled-as.sat03-119.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3262794-1304094214/watcher-3262794-1304094214 -o /tmp/evaluation-result-3262794-1304094214/solver-3262794-1304094214 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262794-1304094214.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= cf96a8eea16f98b3cec72117cedd0285
RANDOM SEED=2077176773

node134.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 7186.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25016640 kB
Buffers:       1490092 kB
Cached:        5856908 kB
SwapCached:         64 kB
Active:        1465932 kB
Inactive:      5946616 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25016640 kB
SwapTotal:    67111528 kB
SwapFree:     67111352 kB
Dirty:            2000 kB
Writeback:           0 kB
AnonPages:       65452 kB
Mapped:          15356 kB
Slab:           459124 kB
PageTables:       4832 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   220564 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71584 MiB
End job on node134 at 2011-04-29 18:43:45