Trace number 3257866

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.34 1203.69

General information on the benchmark

NameCRAFTED/rbsat/random/unforced/
rbsat-v1150c84314g1.cnf
MD5SUMeded74bffb721f48c45af6ff95302642
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark39.8949
Satisfiable
(Un)Satisfiability was proved
Number of variables1150
Number of clauses84314
Sum of the clauses size169678
Maximum clause length23
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 284264
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 550

Solver Data 

0.00/0.05	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3257866-1304047650.cnf 
0.00/0.05	c Using SATTIMEOUT of 4800
0.00/0.05	c Using SATRAM of 15500
0.00/0.05	c Size of subset node = 32
0.00/0.05	c Variables = 1150
0.00/0.05	c Clauses = 84314
0.00/0.05	c Literals = 169678
0.00/0.05	c Reduced size:
0.00/0.05	c Variables = 1150
0.00/0.05	c Clauses = 84314
0.00/0.05	c Literals = 169678
0.00/0.05	c a2
0.09/0.10	c a1
0.19/0.20	c Independent sub-problems = 1
0.19/0.20	c Sub-problem with 1150 variables and 84314 clauses
0.19/0.20	c using permutation method 4
0.19/0.20	c count = 1
0.19/0.20	c Ordering finished 0.180 seconds
0.19/0.20	c mapped 0.190 seconds
0.19/0.20	c permuted 0.190 seconds
0.19/0.20	c about to sort
0.19/0.21	c sorted
0.29/0.38	c clauses now 84314
0.29/0.38	c init finished 0.230 seconds
0.29/0.38	c tree built 0.370 seconds
0.29/0.38	c 
0.29/0.38	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.29/0.38	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.29/0.38	c ................................ | ..........                    |                                  0.4         0.0             
0.29/0.38	c ................................ | ..........                  1 |                                  0.4         2.7             
0.29/0.38	c ................................ | ..........                  2 |                                  0.4         5.4             
0.29/0.38	c ................................ | ..........                  3 |                                  0.4         8.1             
0.29/0.38	c ................................ | ..........                  4 |                                  0.4        10.8             
0.29/0.38	c ................................ | ..........                  5 |                                  0.4        13.5             
0.29/0.39	c ................................ | ..........               1000 |              112                 0.4      2631.6         4747
95.88/24.33	c ................................ | ..........            3515000 | 99829        416870             96.0     36603.1     21019930
191.75/48.37	c ................................ | ..........            6025000 | 178815       717567            192.0     31376.9     33965497
287.98/72.40	c ................................ | ..........            8485000 | 251692       969785            288.0     29461.8     45070540
383.88/96.44	c ................................ | ..........           10506000 | 320899       1189243           384.0     27358.7     51929500
479.70/120.47	c ................................ | *.........           12398000 | 386978       1411705           480.0     25829.2     60900564
575.96/144.51	c ................................ | *.........           14236000 | 454210       1609307           576.0     24714.4     70975027
671.81/168.56	c ................................ | *.........           15984000 | 517687       1798209           672.0     23783.9     77185738
767.65/192.59	c ................................ | *.........           17781000 | 574635       1990052           768.0     23151.7     85263368
863.89/216.63	c ................................ | *.........           19476000 | 630070       2164713           864.0     22540.9     93753424
959.74/240.66	c ................................ | **........           21085000 | 680946       2324554           960.0     21963.5     96853611
1056.55/264.91	c ................................ | **........           22530000 | 729301       2506247          1056.6     21323.1    101774252
1151.73/288.87	c ................................ | **........           23935000 | 775594       2680472          1152.0     20776.7    106097192
1247.70/312.99	c ................................ | **........           25328000 | 817514       2846944          1248.1     20293.9    111756514
1343.70/337.07	c ................................ | **........           26570000 | 856964       2996317          1344.0     19769.3    114399497
1439.69/361.17	c ................................ | ***.......           27856000 | 901042       3137321          1440.0     19344.3    115254592
1535.72/385.27	c ................................ | ***.......           29151000 | 950093       3298438          1536.0     18978.0    116284322
1631.73/409.37	c ................................ | ***.......           30625000 | 1006316      3461143          1632.0     18765.1    121300723
1727.76/433.47	c ................................ | ***.......           32221000 | 1058794      3616039          1728.0     18646.0    125568523
1823.80/457.55	c ................................ | ***.......           33772000 | 1107437      3763961          1824.0     18515.2    130739782
1919.82/481.65	c ................................ | ****......           35307000 | 1153356      3924346          1920.0     18388.8    139541447
2015.85/505.74	c ................................ | ****......           36743000 | 1197445      4071987          2016.0     18225.3    145686614
2111.73/529.78	c ................................ | ****......           38074000 | 1241237      4213389          2112.0     18027.2    148892633
2207.95/553.82	c ................................ | ****......           39404000 | 1284377      4360640          2208.1     17845.5    154575980
2303.80/577.86	c ................................ | ****......           40726000 | 1327723      4502220          2304.0     17675.9    160783609
2400.01/601.90	c ................................ | *****.....           42034000 | 1367118      4639474          2400.0     17513.9    162686132
2495.84/625.94	c ................................ | *****.....           43403000 | 1416033      4774150          2496.0     17389.0    166302310
2591.69/649.99	c ................................ | *****.....           44834000 | 1463211      4905267          2592.1     17296.6    172410570
2687.93/674.03	c ................................ | *****.....           46176000 | 1508705      5042140          2688.1     17178.1    179057808
2783.67/698.08	c ................................ | *****.....           47409000 | 1549086      5162901          2784.0     17029.1    179664921
2879.74/722.18	c ................................ | ******....           48660000 | 1592459      5285364          2880.1     16895.4    179664921
2975.80/746.25	c ................................ | ******....           49891000 | 1635782      5406089          2976.0     16764.3    180891222
3071.87/770.33	c ................................ | ******....           51065000 | 1676232      5526800          3072.0     16622.7    184497482
3167.95/794.43	c ................................ | ******....           52231000 | 1720577      5640064          3168.1     16486.7    189471160
3263.75/818.47	c ................................ | ******....           53359000 | 1762093      5755349          3264.0     16347.6    193225501
3359.99/842.51	c ................................ | *******...           54527000 | 1801951      5862653          3360.0     16228.1    196818988
3455.83/866.55	c ................................ | *******...           55625000 | 1841670      5967665          3456.1     16094.9    200057126
3551.70/890.60	c ................................ | *******...           56676000 | 1879789      6069359          3552.1     15955.7    200057126
3647.90/914.64	c ................................ | *******...           57655000 | 1917651      6188682          3648.1     15804.2    201235771
3743.73/938.67	c ................................ | *******...           58703000 | 1956738      6298428          3744.0     15679.1    204866173
3839.94/962.73	c ................................ | ********..           59835000 | 1994609      6409133          3840.1     15581.7    205072180
3935.79/986.75	c ................................ | ********..           60957000 | 2032488      6518996          3936.0     15487.0    205982769
4031.65/1010.78	c ................................ | ********..           62094000 | 2072298      6619939          4032.0     15400.3    208053726
4127.92/1034.83	c ................................ | ********..           63159000 | 2110357      6721745          4128.1     15300.0    210561287
4223.78/1058.88	c ................................ | ********..           64209000 | 2145031      6830264          4224.1     15200.7    213898274
4319.99/1082.91	c ................................ | *********.           65225000 | 2180641      6933007          4320.0     15098.2    215021326
4416.06/1107.00	c ................................ | *********.           66265000 | 2216336      7041598          4416.1     15005.5    219223520
4512.06/1131.10	c ................................ | *********.           67334000 | 2250450      7146816          4512.1     14923.1    222194000
4607.71/1155.17	c ................................ | *********.           68300000 | 2283557      7249456          4608.0     14822.0    222194000
4703.74/1179.26	c ................................ | *********.           69291000 | 2316271      7362209          4704.0     14730.2    224759325
4799.71/1203.38	c CPU limit (4800) reached
4799.71/1203.38	c Time = 4800.02 seconds
4799.71/1203.38	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3257866-1304047650/watcher-3257866-1304047650 -o /tmp/evaluation-result-3257866-1304047650/solver-3257866-1304047650 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3257866-1304047650.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.96 8.15 8.16 5/185 18520
/proc/meminfo: memFree=23661256/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=9760 CPUtime=0 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) R 18518 18520 18211 0 -1 4202496 496 0 0 0 0 0 0 0 25 0 1 0 516363189 9994240 438 33554432000 4194304 4338100 140734835055792 18446744073709551615 232335553221 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/18520/statm: 2440 439 147 36 0 345 0

[startup+0.0093651 s]
/proc/loadavg: 7.96 8.15 8.16 5/185 18520
/proc/meminfo: memFree=23661256/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=10432 CPUtime=0 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) R 18518 18520 18211 0 -1 4202496 659 0 0 0 0 0 0 0 25 0 1 0 516363189 10682368 601 33554432000 4194304 4338100 140734835055792 18446744073709551615 232335553221 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/18520/statm: 2608 602 147 36 0 513 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 10432

[startup+0.100413 s]
/proc/loadavg: 7.96 8.15 8.16 5/185 18520
/proc/meminfo: memFree=23661256/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=25580 CPUtime=0.09 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) R 18518 18520 18211 0 -1 4202496 4456 0 0 0 9 0 0 0 25 0 1 0 516363189 26193920 4397 33554432000 4194304 4338100 140734835055792 18446744073709551615 232335287312 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/18520/statm: 6395 4397 153 36 0 4300 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 25580

[startup+0.300381 s]
/proc/loadavg: 7.96 8.15 8.16 5/185 18520
/proc/meminfo: memFree=23661256/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=48316 CPUtime=0.29 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) R 18518 18520 18211 0 -1 4202496 12019 0 0 0 28 1 0 0 25 0 1 0 516363189 49475584 9957 33554432000 4194304 4338100 140734835055792 18446744073709551615 232335553221 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/18520/statm: 12079 9964 164 36 0 9984 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 48316

[startup+0.714429 s]
/proc/loadavg: 7.96 8.15 8.16 5/185 18520
/proc/meminfo: memFree=23661256/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=281736 CPUtime=3.61 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) S 18518 18520 18211 0 -1 4202496 26270 0 0 0 352 9 0 0 25 0 5 0 516363189 288497664 24042 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347695077 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 70434 24043 176 36 0 68339 0
Current children cumulated CPU time (s) 3.61
Current children cumulated vsize (KiB) 281736

[startup+1.5001 s]
/proc/loadavg: 7.96 8.15 8.16 5/185 18520
/proc/meminfo: memFree=23661256/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=294380 CPUtime=4.82 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) S 18518 18520 18211 0 -1 4202496 29338 0 0 0 470 12 0 0 25 0 5 0 516363189 301445120 27110 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347695077 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 73595 27111 176 36 0 71500 0
Current children cumulated CPU time (s) 4.82
Current children cumulated vsize (KiB) 294380

[startup+3.10082 s]
/proc/loadavg: 7.96 8.15 8.16 9/190 18525
/proc/meminfo: memFree=23442364/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=360464 CPUtime=11.2 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) S 18518 18520 18211 0 -1 4202496 45595 0 0 0 1091 29 0 0 25 0 5 0 516363189 369115136 43367 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347695077 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 90116 43368 176 36 0 88021 0
[pid=18520/tid=18522] ppid=18518 vsize=360464 CPUtime=2.7 cores=0,2,4,6
/proc/18520/task/18522/stat : 18522 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 9718 0 0 0 268 2 0 0 23 0 5 0 516363228 369115136 43368 33554432000 4194304 4338100 140734835055792 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=18520/tid=18523] ppid=18518 vsize=360464 CPUtime=2.7 cores=0,2,4,6
/proc/18520/task/18523/stat : 18523 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 4470 0 0 0 258 12 0 0 23 0 5 0 516363228 369115136 43368 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347718325 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18520/tid=18524] ppid=18518 vsize=360464 CPUtime=2.69 cores=0,2,4,6
/proc/18520/task/18524/stat : 18524 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 10214 0 0 0 268 1 0 0 23 0 5 0 516363228 369115136 43368 33554432000 4194304 4338100 140734835055792 18446744073709551615 232335588907 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18520/tid=18525] ppid=18518 vsize=360464 CPUtime=2.7 cores=0,2,4,6
/proc/18520/task/18525/stat : 18525 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 4923 0 0 0 258 12 0 0 25 0 5 0 516363228 369115136 43368 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347718325 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 11.2
Current children cumulated vsize (KiB) 360464

[startup+6.30125 s]
/proc/loadavg: 7.96 8.15 8.16 9/190 18525
/proc/meminfo: memFree=23262068/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=468376 CPUtime=23.99 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) S 18518 18520 18211 0 -1 4202496 73516 0 0 0 2339 60 0 0 25 0 5 0 516363189 479617024 71288 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347695077 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 117094 71288 176 36 0 114999 0
[pid=18520/tid=18522] ppid=18518 vsize=468376 CPUtime=5.9 cores=0,2,4,6
/proc/18520/task/18522/stat : 18522 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 19532 0 0 0 585 5 0 0 25 0 5 0 516363228 479617024 71288 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=18520/tid=18523] ppid=18518 vsize=468376 CPUtime=5.9 cores=0,2,4,6
/proc/18520/task/18523/stat : 18523 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 8668 0 0 0 566 24 0 0 25 0 5 0 516363228 479617024 71288 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347718325 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18520/tid=18524] ppid=18518 vsize=468376 CPUtime=5.87 cores=0,2,4,6
/proc/18520/task/18524/stat : 18524 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 19541 0 0 0 585 2 0 0 25 0 5 0 516363228 479617024 71288 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18520/tid=18525] ppid=18518 vsize=468376 CPUtime=5.9 cores=0,2,4,6
/proc/18520/task/18525/stat : 18525 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 9504 0 0 0 565 25 0 0 25 0 5 0 516363228 479617024 71288 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347718325 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 23.99
Current children cumulated vsize (KiB) 468376

[startup+12.7011 s]
/proc/loadavg: 7.97 8.15 8.16 9/190 18526
/proc/meminfo: memFree=22988916/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=660144 CPUtime=49.57 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) S 18518 18520 18211 0 -1 4202496 120632 0 0 0 4838 119 0 0 25 0 5 0 516363189 675987456 118404 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347695077 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 165036 118404 176 36 0 162941 0
[pid=18520/tid=18522] ppid=18518 vsize=660144 CPUtime=12.31 cores=0,2,4,6
/proc/18520/task/18522/stat : 18522 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 35291 0 0 0 1222 9 0 0 25 0 5 0 516363228 675987456 118404 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=18520/tid=18523] ppid=18518 vsize=660144 CPUtime=12.3 cores=0,2,4,6
/proc/18520/task/18523/stat : 18523 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 15346 0 0 0 1179 51 0 0 25 0 5 0 516363228 675987456 118404 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347718325 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18520/tid=18524] ppid=18518 vsize=660144 CPUtime=12.24 cores=0,2,4,6
/proc/18520/task/18524/stat : 18524 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 37216 0 0 0 1219 5 0 0 25 0 5 0 516363228 675987456 118404 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254342 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18520/tid=18525] ppid=18518 vsize=660144 CPUtime=12.3 cores=0,2,4,6
/proc/18520/task/18525/stat : 18525 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 16508 0 0 0 1179 51 0 0 25 0 5 0 516363228 675987456 118404 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347718325 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 49.57
Current children cumulated vsize (KiB) 660144

[startup+25.5008 s]
/proc/loadavg: 8.11 8.17 8.16 9/190 18526
/proc/meminfo: memFree=22650148/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=937560 CPUtime=100.68 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) S 18518 18520 18211 0 -1 4202496 190539 0 0 0 9833 235 0 0 25 0 5 0 516363189 960061440 187063 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347695077 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 234390 187063 176 36 0 232295 0
[pid=18520/tid=18522] ppid=18518 vsize=937560 CPUtime=25.1 cores=0,2,4,6
/proc/18520/task/18522/stat : 18522 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 58871 0 0 0 2495 15 0 0 25 0 5 0 516363228 960061440 187063 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254321 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=18520/tid=18523] ppid=18518 vsize=937560 CPUtime=25.1 cores=0,2,4,6
/proc/18520/task/18523/stat : 18523 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 25491 0 0 0 2404 106 0 0 25 0 5 0 516363228 960061440 187063 33554432000 4194304 4338100 140734835055792 18446744073709551615 232335287742 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18520/tid=18524] ppid=18518 vsize=937560 CPUtime=24.97 cores=0,2,4,6
/proc/18520/task/18524/stat : 18524 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 64429 0 0 0 2487 10 0 0 25 0 5 0 516363228 960061440 187063 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18520/tid=18525] ppid=18518 vsize=937560 CPUtime=25.1 cores=0,2,4,6
/proc/18520/task/18525/stat : 18525 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 25477 0 0 0 2409 101 0 0 25 0 5 0 516363228 960061440 187063 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347718325 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 100.68
Current children cumulated vsize (KiB) 937560

[startup+51.1003 s]
/proc/loadavg: 8.07 8.16 8.16 9/190 18527
/proc/meminfo: memFree=21840008/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=1372352 CPUtime=202.93 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) S 18518 18520 18211 0 -1 4202496 298906 0 0 0 19848 445 0 0 25 0 5 0 516363189 1405288448 295430 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347695077 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 343088 295430 176 36 0 340993 0
[pid=18520/tid=18522] ppid=18518 vsize=1372352 CPUtime=50.7 cores=0,2,4,6
/proc/18520/task/18522/stat : 18522 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 97100 0 0 0 5046 24 0 0 25 0 5 0 516363228 1405288448 295430 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=18520/tid=18523] ppid=18518 vsize=1372352 CPUtime=50.69 cores=0,2,4,6
/proc/18520/task/18523/stat : 18523 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 39353 0 0 0 4866 203 0 0 25 0 5 0 516363228 1405288448 295430 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347718325 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18520/tid=18524] ppid=18518 vsize=1372352 CPUtime=50.43 cores=0,2,4,6
/proc/18520/task/18524/stat : 18524 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 107534 0 0 0 5025 18 0 0 25 0 5 0 516363228 1405288448 295430 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18520/tid=18525] ppid=18518 vsize=1372352 CPUtime=50.7 cores=0,2,4,6
/proc/18520/task/18525/stat : 18525 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 38648 0 0 0 4873 197 0 0 25 0 5 0 516363228 1405288448 295430 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347718325 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 202.93
Current children cumulated vsize (KiB) 1372352

[startup+102.3 s]
/proc/loadavg: 8.07 8.15 8.16 9/190 18528
/proc/meminfo: memFree=20662500/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=2019668 CPUtime=407.41 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) S 18518 18520 18211 0 -1 4202496 456366 0 0 0 39934 807 0 0 25 0 5 0 516363189 2068140032 452890 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347695077 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 504917 452890 176 36 0 502822 0
[pid=18520/tid=18522] ppid=18518 vsize=2019668 CPUtime=101.89 cores=0,2,4,6
/proc/18520/task/18522/stat : 18522 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 154400 0 0 0 10148 41 0 0 25 0 5 0 516363228 2068140032 452890 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=18520/tid=18523] ppid=18518 vsize=2019668 CPUtime=101.78 cores=0,2,4,6
/proc/18520/task/18523/stat : 18523 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 54965 0 0 0 9806 372 0 0 25 0 5 0 516363228 2068140032 452890 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18520/tid=18524] ppid=18518 vsize=2019668 CPUtime=101.43 cores=0,2,4,6
/proc/18520/task/18524/stat : 18524 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 176354 0 0 0 10111 32 0 0 25 0 5 0 516363228 2068140032 452891 33554432000 4194304 4338100 140734835055792 18446744073709551615 4263461 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18520/tid=18525] ppid=18518 vsize=2019668 CPUtime=101.88 cores=0,2,4,6
/proc/18520/task/18525/stat : 18525 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 54376 0 0 0 9830 358 0 0 25 0 5 0 516363228 2068140032 452891 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 407.41
Current children cumulated vsize (KiB) 2019668

[startup+162.301 s]
/proc/loadavg: 8.12 8.15 8.15 9/190 18530
/proc/meminfo: memFree=19426828/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=2661700 CPUtime=647.07 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) S 18518 18520 18211 0 -1 4202496 618308 0 0 0 63516 1191 0 0 25 0 5 0 516363189 2725580800 614832 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347695077 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 665425 614832 176 36 0 663330 0
[pid=18520/tid=18522] ppid=18518 vsize=2661700 CPUtime=161.89 cores=0,2,4,6
/proc/18520/task/18522/stat : 18522 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 208109 0 0 0 16132 57 0 0 25 0 5 0 516363228 2725580800 614832 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=18520/tid=18523] ppid=18518 vsize=2661700 CPUtime=161.44 cores=0,2,4,6
/proc/18520/task/18523/stat : 18523 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 71973 0 0 0 15596 548 0 0 25 0 5 0 516363228 2725580800 614832 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18520/tid=18524] ppid=18518 vsize=2661700 CPUtime=161.44 cores=0,2,4,6
/proc/18520/task/18524/stat : 18524 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 254007 0 0 0 16097 47 0 0 25 0 5 0 516363228 2725580800 614832 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18520/tid=18525] ppid=18518 vsize=2661700 CPUtime=161.87 cores=0,2,4,6
/proc/18520/task/18525/stat : 18525 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 67948 0 0 0 15652 535 0 0 25 0 5 0 516363228 2725580800 614832 33554432000 4194304 4338100 140734835055792 18446744073709551615 232335287348 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 647.07
Current children cumulated vsize (KiB) 2661700

[startup+222.301 s]
/proc/loadavg: 8.05 8.12 8.14 9/190 18531
/proc/meminfo: memFree=18381860/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=3250424 CPUtime=886.66 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) S 18518 18520 18211 0 -1 4202496 765761 0 0 0 87107 1559 0 0 25 0 5 0 516363189 3328434176 762285 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347695077 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 812606 762285 176 36 0 810511 0
[pid=18520/tid=18522] ppid=18518 vsize=3250424 CPUtime=221.86 cores=0,2,4,6
/proc/18520/task/18522/stat : 18522 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 263644 0 0 0 22113 73 0 0 25 0 5 0 516363228 3328434176 762285 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254041 0 0 4096 0 0 0 0 -1 0 0 0 0

################
# More data... #
################

/proc/18520/task/18522/stat : 18522 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 592801 0 0 0 75831 178 0 0 25 0 5 0 516363228 6326382592 1489294 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18520/tid=18523] ppid=18518 vsize=6178108 CPUtime=758.19 cores=0,2,4,6
/proc/18520/task/18523/stat : 18523 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 145551 0 0 0 74449 1370 0 0 25 0 5 0 516363228 6326382592 1489294 33554432000 4194304 4338100 140734835055792 18446744073709551615 4253954 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=18520/tid=18524] ppid=18518 vsize=6178108 CPUtime=760.36 cores=0,2,4,6
/proc/18520/task/18524/stat : 18524 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 596108 0 0 0 75921 115 0 0 25 0 5 0 516363228 6326382592 1489294 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18520/tid=18525] ppid=18518 vsize=6178108 CPUtime=760.93 cores=0,2,4,6
/proc/18520/task/18525/stat : 18525 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 142039 0 0 0 74548 1545 0 0 18 0 5 0 516363228 6326382592 1489294 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347718325 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3039.99
Current children cumulated vsize (KiB) 6178108

[startup+822.3 s]
/proc/loadavg: 8.10 8.17 8.17 9/190 18547
/proc/meminfo: memFree=12365460/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=6497428 CPUtime=3279.33 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) S 18518 18520 18211 0 -1 4202496 1573252 0 0 0 324577 3356 0 0 25 0 5 0 516363189 6653366272 1569776 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347695077 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 1624357 1569776 176 36 0 1622262 0
[pid=18520/tid=18522] ppid=18518 vsize=6497428 CPUtime=819.86 cores=0,2,4,6
/proc/18520/task/18522/stat : 18522 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 636248 0 0 0 81795 191 0 0 25 0 5 0 516363228 6653366272 1569776 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=18520/tid=18523] ppid=18518 vsize=6497428 CPUtime=818.08 cores=0,2,4,6
/proc/18520/task/18523/stat : 18523 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 145587 0 0 0 80361 1447 0 0 25 0 5 0 516363228 6653366272 1569776 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18520/tid=18524] ppid=18518 vsize=6497428 CPUtime=820.35 cores=0,2,4,6
/proc/18520/task/18524/stat : 18524 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 633076 0 0 0 81912 123 0 0 25 0 5 0 516363228 6653366272 1569776 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18520/tid=18525] ppid=18518 vsize=6497428 CPUtime=820.62 cores=0,2,4,6
/proc/18520/task/18525/stat : 18525 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 142070 0 0 0 80471 1591 0 0 25 0 5 0 516363228 6653366272 1569776 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3279.33
Current children cumulated vsize (KiB) 6497428

[startup+882.301 s]
/proc/loadavg: 8.27 8.20 8.18 9/190 18549
/proc/meminfo: memFree=11926968/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=6688500 CPUtime=3518.93 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) S 18518 18520 18211 0 -1 4202496 1620369 0 0 0 348280 3613 0 0 25 0 5 0 516363189 6849024000 1616893 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347695077 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 1672125 1616893 176 36 0 1670030 0
[pid=18520/tid=18522] ppid=18518 vsize=6688500 CPUtime=879.84 cores=0,2,4,6
/proc/18520/task/18522/stat : 18522 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 657330 0 0 0 87787 197 0 0 25 0 5 0 516363228 6849024000 1616893 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347705038 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=18520/tid=18523] ppid=18518 vsize=6688500 CPUtime=877.85 cores=0,2,4,6
/proc/18520/task/18523/stat : 18523 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 145620 0 0 0 86217 1568 0 0 25 0 5 0 516363228 6849024000 1616893 33554432000 4194304 4338100 140734835055792 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18520/tid=18524] ppid=18518 vsize=6688500 CPUtime=880.34 cores=0,2,4,6
/proc/18520/task/18524/stat : 18524 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 659044 0 0 0 87903 131 0 0 25 0 5 0 516363228 6849024000 1616893 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254304 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18520/tid=18525] ppid=18518 vsize=6688500 CPUtime=880.47 cores=0,2,4,6
/proc/18520/task/18525/stat : 18525 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 142104 0 0 0 86334 1713 0 0 25 0 5 0 516363228 6849024000 1616893 33554432000 4194304 4338100 140734835055792 18446744073709551615 232335287376 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3518.93
Current children cumulated vsize (KiB) 6688500

[startup+942.307 s]
/proc/loadavg: 8.16 8.19 8.18 9/190 18550
/proc/meminfo: memFree=11669896/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=6910700 CPUtime=3758.51 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) S 18518 18520 18211 0 -1 4202496 1659817 0 0 0 371937 3914 0 0 25 0 5 0 516363189 7076556800 1656341 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347695077 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 1727675 1656341 176 36 0 1725580 0
[pid=18520/tid=18522] ppid=18518 vsize=6910700 CPUtime=939.81 cores=0,2,4,6
/proc/18520/task/18522/stat : 18522 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 681662 0 0 0 93775 206 0 0 25 0 5 0 516363228 7076556800 1656341 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=18520/tid=18523] ppid=18518 vsize=6910700 CPUtime=937.83 cores=0,2,4,6
/proc/18520/task/18523/stat : 18523 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 145676 0 0 0 92072 1711 0 0 25 0 5 0 516363228 7076556800 1656341 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18520/tid=18524] ppid=18518 vsize=6910700 CPUtime=940.33 cores=0,2,4,6
/proc/18520/task/18524/stat : 18524 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 674044 0 0 0 93898 135 0 0 25 0 5 0 516363228 7076556800 1656341 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18520/tid=18525] ppid=18518 vsize=6910700 CPUtime=940.12 cores=0,2,4,6
/proc/18520/task/18525/stat : 18525 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 142164 0 0 0 92153 1859 0 0 25 0 5 0 516363228 7076556800 1656341 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3758.51
Current children cumulated vsize (KiB) 6910700

[startup+1002.3 s]
/proc/loadavg: 8.30 8.21 8.19 9/190 18552
/proc/meminfo: memFree=11566948/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=6989164 CPUtime=3998.1 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) S 18518 18520 18211 0 -1 4202496 1679242 0 0 0 395636 4174 0 0 25 0 5 0 516363189 7156903936 1675766 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347695077 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 1747291 1675766 176 36 0 1745196 0
[pid=18520/tid=18522] ppid=18518 vsize=6989164 CPUtime=999.79 cores=0,2,4,6
/proc/18520/task/18522/stat : 18522 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 692910 0 0 0 99768 211 0 0 25 0 5 0 516363228 7156903936 1675766 33554432000 4194304 4338100 140734835055792 18446744073709551615 4234377 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=18520/tid=18523] ppid=18518 vsize=6989164 CPUtime=997.83 cores=0,2,4,6
/proc/18520/task/18523/stat : 18523 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 145711 0 0 0 97944 1839 0 0 25 0 5 0 516363228 7156903936 1675766 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18520/tid=18524] ppid=18518 vsize=6989164 CPUtime=1000.29 cores=0,2,4,6
/proc/18520/task/18524/stat : 18524 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 682152 0 0 0 99892 137 0 0 25 0 5 0 516363228 7156903936 1675766 33554432000 4194304 4338100 140734835055792 18446744073709551615 4239444 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18520/tid=18525] ppid=18518 vsize=6989164 CPUtime=999.78 cores=0,2,4,6
/proc/18520/task/18525/stat : 18525 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 142198 0 0 0 97994 1984 0 0 25 0 5 0 516363228 7156903936 1675766 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3998.1
Current children cumulated vsize (KiB) 6989164

[startup+1062.3 s]
/proc/loadavg: 8.18 8.19 8.18 9/190 18554
/proc/meminfo: memFree=11186136/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=7195876 CPUtime=4237.74 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) S 18518 18520 18211 0 -1 4202496 1731540 0 0 0 419326 4448 0 0 25 0 5 0 516363189 7368577024 1728064 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347695077 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 1798969 1728064 176 36 0 1796874 0
[pid=18520/tid=18522] ppid=18518 vsize=7195876 CPUtime=1059.79 cores=0,2,4,6
/proc/18520/task/18522/stat : 18522 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 720004 0 0 0 105759 220 0 0 25 0 5 0 516363228 7368577024 1728064 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254332 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=18520/tid=18523] ppid=18518 vsize=7195876 CPUtime=1057.82 cores=0,2,4,6
/proc/18520/task/18523/stat : 18523 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 145755 0 0 0 103816 1966 0 0 25 0 5 0 516363228 7368577024 1728064 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18520/tid=18524] ppid=18518 vsize=7195876 CPUtime=1060.27 cores=0,2,4,6
/proc/18520/task/18524/stat : 18524 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 707266 0 0 0 105884 143 0 0 25 0 5 0 516363228 7368577024 1728064 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18520/tid=18525] ppid=18518 vsize=7195876 CPUtime=1059.45 cores=0,2,4,6
/proc/18520/task/18525/stat : 18525 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 142244 0 0 0 103830 2115 0 0 25 0 5 0 516363228 7368577024 1728064 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4237.74
Current children cumulated vsize (KiB) 7195876

[startup+1122.3 s]
/proc/loadavg: 7.57 8.05 8.13 5/180 18573
/proc/meminfo: memFree=18264280/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=7430740 CPUtime=4477.01 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) S 18518 18520 18211 0 -1 4202496 1790004 0 0 0 443131 4570 0 0 25 0 5 0 516363189 7609077760 1786528 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347695077 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 1857685 1786528 176 36 0 1855590 0
[pid=18520/tid=18522] ppid=18518 vsize=7430740 CPUtime=1119.51 cores=0,2,4,6
/proc/18520/task/18522/stat : 18522 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 751250 0 0 0 111720 231 0 0 25 0 5 0 516363228 7609077760 1786528 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254321 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18520/tid=18523] ppid=18518 vsize=7430740 CPUtime=1117.49 cores=0,2,4,6
/proc/18520/task/18523/stat : 18523 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 145793 0 0 0 109748 2001 0 0 18 0 5 0 516363228 7609077760 1786528 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18520/tid=18524] ppid=18518 vsize=7430740 CPUtime=1120.25 cores=0,2,4,6
/proc/18520/task/18524/stat : 18524 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 734407 0 0 0 111876 149 0 0 25 0 5 0 516363228 7609077760 1786528 33554432000 4194304 4338100 140734835055792 18446744073709551615 4288007 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18520/tid=18525] ppid=18518 vsize=7430740 CPUtime=1119.34 cores=0,2,4,6
/proc/18520/task/18525/stat : 18525 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 142283 0 0 0 109748 2186 0 0 25 0 5 0 516363228 7609077760 1786528 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347718325 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4477.01
Current children cumulated vsize (KiB) 7430740

[startup+1182.3 s]
/proc/loadavg: 5.41 7.34 7.88 5/180 18575
/proc/meminfo: memFree=18155432/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=7534228 CPUtime=4716.07 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) S 18518 18520 18211 0 -1 4202496 1816425 0 0 0 466979 4628 0 0 25 0 5 0 516363189 7715049472 1812949 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347695077 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 1883557 1812949 176 36 0 1881462 0
[pid=18520/tid=18522] ppid=18518 vsize=7534228 CPUtime=1179.14 cores=0,2,4,6
/proc/18520/task/18522/stat : 18522 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 763005 0 0 0 117678 236 0 0 25 0 5 0 516363228 7715049472 1812949 33554432000 4194304 4338100 140734835055792 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=18520/tid=18523] ppid=18518 vsize=7534228 CPUtime=1176.96 cores=0,2,4,6
/proc/18520/task/18523/stat : 18523 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 145836 0 0 0 115692 2004 0 0 19 0 5 0 516363228 7715049472 1812949 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347718325 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=18520/tid=18524] ppid=18518 vsize=7534228 CPUtime=1180.24 cores=0,2,4,6
/proc/18520/task/18524/stat : 18524 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 748997 0 0 0 117871 153 0 0 25 0 5 0 516363228 7715049472 1812949 33554432000 4194304 4338100 140734835055792 18446744073709551615 4284214 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=18520/tid=18525] ppid=18518 vsize=7534228 CPUtime=1179.31 cores=0,2,4,6
/proc/18520/task/18525/stat : 18525 (ttsth-5-4) R 18518 18520 18211 0 -1 4202560 142316 0 0 0 115700 2231 0 0 25 0 5 0 516363228 7715049472 1812949 33554432000 4194304 4338100 140734835055792 18446744073709551615 232347718325 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4716.07
Current children cumulated vsize (KiB) 7534228



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.4 s]
/proc/loadavg: 5.01 7.12 7.80 2/177 18575
/proc/meminfo: memFree=18538716/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=0 CPUtime=4800.04 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) Z 18518 18520 18211 0 -1 4203548 1822666 0 0 0 475352 4652 0 0 18 0 2 0 516363189 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 0 0 0 0 0 0 0
[pid=18520/tid=18525] ppid=18518 vsize=0 CPUtime=1200.39 cores=0,2,4,6
/proc/18520/task/18525/stat : 18525 (ttsth-5-4) R 18518 18520 18211 0 -1 4203588 142331 0 0 0 117787 2252 0 0 25 0 2 0 516363228 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.04
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.51 s]
/proc/loadavg: 5.01 7.12 7.80 2/177 18575
/proc/meminfo: memFree=18538716/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=0 CPUtime=4800.15 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) Z 18518 18520 18211 0 -1 4203548 1822666 0 0 0 475352 4663 0 0 18 0 2 0 516363189 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 0 0 0 0 0 0 0
[pid=18520/tid=18525] ppid=18518 vsize=0 CPUtime=1200.49 cores=0,2,4,6
/proc/18520/task/18525/stat : 18525 (ttsth-5-4) R 18518 18520 18211 0 -1 4203588 142331 0 0 0 117787 2262 0 0 25 0 2 0 516363228 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.15
Current children cumulated vsize (KiB) 0

[startup+1203.6 s]
/proc/loadavg: 5.01 7.12 7.80 2/177 18575
/proc/meminfo: memFree=18538716/32951124 swapFree=67111524/67111528
[pid=18520] ppid=18518 vsize=0 CPUtime=4800.24 cores=0,2,4,6
/proc/18520/stat : 18520 (ttsth-5-4) Z 18518 18520 18211 0 -1 4203548 1822666 0 0 0 475352 4672 0 0 18 0 2 0 516363189 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/18520/statm: 0 0 0 0 0 0 0
[pid=18520/tid=18525] ppid=18518 vsize=0 CPUtime=1200.58 cores=0,2,4,6
/proc/18520/task/18525/stat : 18525 (ttsth-5-4) R 18518 18520 18211 0 -1 4203588 142331 0 0 0 117787 2271 0 0 25 0 2 0 516363228 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.24
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1203.69
CPU time (s): 4800.34
CPU user time (s): 4753.53
CPU system time (s): 46.8099
CPU usage (%): 398.802
Max. virtual memory (cumulated for all children) (KiB): 7559252

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4753.53
system time used= 46.8099
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1822666
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 9965
involuntary context switches= 81405

runsolver used 1.91971 second user time and 4.65329 second system time

The end

Launcher Data 

Begin job on node132 at 2011-04-29 05:27:30
IDJOB=3257866
IDBENCH=71273
IDSOLVER=1654
FILE ID=node132/3257866-1304047650
RUNJOBID= node132-1304045149-18229
PBS_JOBID= 13170396
Free space on /tmp= 71872 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT09/CRAFTED/rbsat/random/unforced/rbsat-v1150c84314g1.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3257866-1304047650/watcher-3257866-1304047650 -o /tmp/evaluation-result-3257866-1304047650/solver-3257866-1304047650 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3257866-1304047650.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= eded74bffb721f48c45af6ff95302642
RANDOM SEED=827934824

node132.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23661404 kB
Buffers:        760656 kB
Cached:        6209272 kB
SwapCached:          4 kB
Active:        3615376 kB
Inactive:      5174372 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23661404 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:            1160 kB
Writeback:           0 kB
AnonPages:     1819200 kB
Mapped:          15208 kB
Slab:           434936 kB
PageTables:       8140 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  2022000 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71872 MiB
End job on node132 at 2011-04-29 05:47:36