Trace number 3258526

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.23 1204.07

General information on the benchmark

NameCRAFTED/rbsat/random/
forced/rbsat-v2100c215164gyes7.cnf
MD5SUM140eea74f7e66501965365160d02fa4d
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2100
Number of clauses215164
Sum of the clauses size432288
Maximum clause length30
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2215094
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 570

Solver Data 

0.09/0.14	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3258526-1304051825.cnf 
0.09/0.14	c Using SATTIMEOUT of 4800
0.09/0.14	c Using SATRAM of 15500
0.09/0.14	c Size of subset node = 32
0.09/0.14	c Variables = 2100
0.09/0.14	c Clauses = 215164
0.09/0.14	c Literals = 432288
0.09/0.14	c Reduced size:
0.09/0.14	c Variables = 2100
0.09/0.14	c Clauses = 215164
0.09/0.14	c Literals = 432288
0.09/0.14	c a2
0.19/0.27	c a1
0.49/0.55	c Independent sub-problems = 1
0.49/0.55	c Sub-problem with 2100 variables and 215164 clauses
0.49/0.55	c using permutation method 4
0.49/0.55	c count = 1
0.49/0.55	c Ordering finished 0.510 seconds
0.49/0.55	c mapped 0.540 seconds
0.49/0.55	c permuted 0.540 seconds
0.49/0.55	c about to sort
0.49/0.57	c sorted
0.99/1.09	c clauses now 215164
0.99/1.09	c init finished 0.610 seconds
0.99/1.09	c tree built 1.080 seconds
0.99/1.09	c 
0.99/1.09	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.99/1.09	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.99/1.09	c ................................ | ..........                    |                                  1.1         0.0             
0.99/1.09	c ................................ | ..........                  1 |                                  1.1         0.9             
0.99/1.09	c ................................ | ..........                  2 |                                  1.1         1.9             
0.99/1.09	c ................................ | ..........                  3 |                                  1.1         2.8             
0.99/1.09	c ................................ | ..........                  4 |                                  1.1         3.7             
0.99/1.09	c ................................ | ..........                  5 |                                  1.1         4.6             
0.99/1.10	c ................................ | ..........               1000 |              134                 1.1       909.1         2685
95.76/24.96	c ................................ | ..........            3080000 | 77596        268895             96.0     32083.3     17782029
191.72/49.07	c ................................ | ..........            5846000 | 143655       471311            192.0     30447.9     29803547
287.72/73.17	c ................................ | ..........            8373000 | 200625       623833            288.0     29072.9     39016682
383.70/97.28	c ................................ | ..........           10363000 | 255131       790369            384.0     26985.6     48197085
479.73/121.37	c ................................ | *.........           12506000 | 307036       935997            480.0     26052.5     54233390
575.76/145.45	c ................................ | *.........           15165000 | 360053       1094938           576.0     26328.1     62690869
671.79/169.56	c ................................ | *.........           17597000 | 408925       1262883           672.0     26184.8     69853577
767.79/193.65	c ................................ | *.........           19684000 | 460667       1403066           768.0     25630.2     76310821
863.85/217.75	c ................................ | *.........           21671000 | 508093       1536948           864.0     25080.7     84109222
959.91/241.82	c ................................ | **........           23638000 | 553018       1656996           960.0     24622.7     91435402
1055.90/265.92	c ................................ | **........           25431000 | 598621       1778579          1056.0     24082.2     92226068
1151.94/290.02	c ................................ | **........           27120000 | 637709       1896816          1152.0     23540.8     92226068
1247.62/314.09	c ................................ | **........           28764000 | 680471       1996895          1248.0     23047.7     95251629
1343.72/338.17	c ................................ | **........           30603000 | 721492       2106035          1344.0     22769.8    100629761
1439.73/362.27	c ................................ | ***.......           32114000 | 760674       2201558          1440.0     22301.1    102194574
1535.80/386.35	c ................................ | ***.......           33616000 | 800488       2298662          1536.0     21885.0    102194574
1631.81/410.45	c ................................ | ***.......           35097000 | 835453       2418234          1632.0     21505.3    102194574
1727.91/434.53	c ................................ | ***.......           36514000 | 871063       2531092          1728.0     21130.7    102194574
1823.97/458.61	c ................................ | ***.......           38299000 | 908710       2645635          1824.0     20997.0    103673304
1920.02/482.70	c ................................ | ****......           40241000 | 947774       2768581          1920.0     20958.4    106058152
2015.74/506.76	c ................................ | ****......           42134000 | 987452       2875778          2016.0     20899.8    109061567
2112.00/530.81	c ................................ | ****......           43931000 | 1020585      2994834          2112.0     20800.3    111578465
2207.86/554.83	c ................................ | ****......           45668000 | 1057338      3110076          2208.0     20683.0    114154405
2303.69/578.88	c ................................ | ****......           47309000 | 1093210      3209654          2304.0     20533.2    116509973
2399.93/602.92	c ................................ | *****.....           48840000 | 1126392      3315116          2400.0     20349.9    116509973
2495.79/626.97	c ................................ | *****.....           50226000 | 1160180      3406445          2496.1     20122.1    116509973
2592.03/651.01	c ................................ | *****.....           51636000 | 1194813      3492133          2592.1     19920.8    116509973
2687.89/675.03	c ................................ | *****.....           53091000 | 1228244      3580386          2688.0     19750.9    116723728
2783.74/699.07	c ................................ | *****.....           54509000 | 1261940      3662112          2784.0     19579.2    118090314
2879.99/723.10	c ................................ | ******....           55946000 | 1294147      3751048          2880.0     19425.6    118630521
2975.85/747.14	c ................................ | ******....           57334000 | 1325777      3831634          2976.0     19265.3    118710596
3071.67/771.18	c ................................ | ******....           58637000 | 1354978      3917501          3072.0     19087.6    120695285
3167.92/795.22	c ................................ | ******....           60005000 | 1388974      3998924          3168.0     18940.8    122561006
3263.78/819.25	c ................................ | ******....           61421000 | 1420646      4082939          3264.0     18817.7    124670626
3359.62/843.29	c ................................ | *******...           62760000 | 1453925      4164128          3360.0     18678.6    124900935
3455.87/867.34	c ................................ | *******...           63947000 | 1481800      4242220          3456.1     18502.9    124900935
3551.73/891.37	c ................................ | *******...           65173000 | 1511520      4320115          3552.1     18347.9    125029533
3647.99/915.40	c ................................ | *******...           66549000 | 1541944      4400679          3648.0     18242.5    128061596
3743.88/939.45	c ................................ | *******...           67759000 | 1571187      4471972          3744.1     18097.7    130244989
3839.74/963.47	c ................................ | ********..           68966000 | 1599350      4554278          3840.0     17959.7    132726849
3935.98/987.51	c ................................ | ********..           70210000 | 1628140      4642311          3936.0     17837.7    137042796
4031.84/1011.55	c ................................ | ********..           71354000 | 1654282      4733261          4032.0     17696.7    141044856
4127.68/1035.59	c ................................ | ********..           72554000 | 1681124      4822598          4128.1     17575.8    145340652
4223.89/1059.62	c ................................ | ********..           73821000 | 1713929      4903547          4224.0     17476.5    150229117
4319.74/1083.68	c ................................ | *********.           75089000 | 1746132      4983825          4320.1     17381.4    154928195
4415.61/1107.70	c ................................ | *********.           76224000 | 1775706      5064590          4416.0     17260.8    156278531
4511.87/1131.75	c ................................ | *********.           77396000 | 1804534      5144443          4512.1     17153.1    156278531
4607.73/1155.78	c ................................ | *********.           78670000 | 1835354      5227979          4608.0     17072.3    157742309
4704.00/1179.81	c ................................ | *********.           80190000 | 1870983      5313909          4704.0     17047.1    160708087
4799.81/1203.84	c CPU limit (4800) reached
4799.81/1203.84	c Time = 4800.00 seconds
4799.81/1203.84	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3258526-1304051825/watcher-3258526-1304051825 -o /tmp/evaluation-result-3258526-1304051825/solver-3258526-1304051825 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3258526-1304051825.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.67 7.49 7.65 2/174 362
/proc/meminfo: memFree=27177004/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=8980 CPUtime=0 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) R 360 362 330 0 -1 4202496 304 0 0 0 0 0 0 0 25 0 1 0 516780979 9195520 247 33554432000 4194304 4338100 140733902431984 18446744073709551615 210204344075 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/362/statm: 2245 251 147 36 0 150 0

[startup+0.014207 s]
/proc/loadavg: 5.67 7.49 7.65 2/174 362
/proc/meminfo: memFree=27177004/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=11296 CPUtime=0.01 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) R 360 362 330 0 -1 4202496 903 0 0 0 1 0 0 0 25 0 1 0 516780979 11567104 845 33554432000 4194304 4338100 140733902431984 18446744073709551615 210204308165 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/362/statm: 2857 846 148 36 0 762 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 11296

[startup+0.100221 s]
/proc/loadavg: 5.67 7.49 7.65 2/174 362
/proc/meminfo: memFree=27177004/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=28676 CPUtime=0.09 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) R 360 362 330 0 -1 4202496 5165 0 0 0 9 0 0 0 25 0 1 0 516780979 29364224 5107 33554432000 4194304 4338100 140733902431984 18446744073709551615 210204308165 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/362/statm: 7169 5109 148 36 0 5074 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 28676

[startup+0.301182 s]
/proc/loadavg: 5.67 7.49 7.65 2/174 362
/proc/meminfo: memFree=27177004/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=53972 CPUtime=0.29 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) R 360 362 330 0 -1 4202496 11792 0 0 0 29 0 0 0 25 0 1 0 516780979 55267328 11312 33554432000 4194304 4338100 140733902431984 18446744073709551615 4211200 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/362/statm: 13493 11312 153 36 0 11398 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 53972

[startup+0.701133 s]
/proc/loadavg: 5.67 7.49 7.65 2/174 362
/proc/meminfo: memFree=27177004/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=81020 CPUtime=0.69 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) R 360 362 330 0 -1 4202496 19389 0 0 0 68 1 0 0 25 0 1 0 516780979 82964480 18067 33554432000 4194304 4338100 140733902431984 18446744073709551615 210204308165 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/362/statm: 20255 18069 164 36 0 18160 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 81020

[startup+1.5009 s]
/proc/loadavg: 5.37 7.40 7.62 3/178 369
/proc/meminfo: memFree=26960220/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=340276 CPUtime=2.69 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) S 360 362 330 0 -1 4202496 40650 0 0 0 264 5 0 0 25 0 5 0 516780979 348442624 38906 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 85069 38906 176 36 0 82974 0
Current children cumulated CPU time (s) 2.69
Current children cumulated vsize (KiB) 340276

[startup+3.10063 s]
/proc/loadavg: 5.37 7.40 7.62 10/186 377
/proc/meminfo: memFree=26811852/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=390852 CPUtime=9.05 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) S 360 362 330 0 -1 4202496 52908 0 0 0 896 9 0 0 25 0 5 0 516780979 400232448 51164 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 97713 51164 176 36 0 95618 0
[pid=362/tid=370] ppid=360 vsize=390852 CPUtime=1.98 cores=0,2,4,6
/proc/362/task/370/stat : 370 (ttsth-5-4) R 360 362 330 0 -1 4202560 5515 0 0 0 197 1 0 0 21 0 5 0 516781089 400232448 51164 33554432000 4194304 4338100 140733902431984 18446744073709551615 210204042636 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=362/tid=371] ppid=360 vsize=390852 CPUtime=1.96 cores=0,2,4,6
/proc/362/task/371/stat : 371 (ttsth-5-4) R 360 362 330 0 -1 4202560 3559 0 0 0 196 0 0 0 18 0 5 0 516781089 400232448 51164 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=362/tid=372] ppid=360 vsize=390852 CPUtime=1.99 cores=0,2,4,6
/proc/362/task/372/stat : 372 (ttsth-5-4) R 360 362 330 0 -1 4202560 5278 0 0 0 199 0 0 0 21 0 5 0 516781089 400232448 51164 33554432000 4194304 4338100 140733902431984 18446744073709551615 4256655 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=362/tid=373] ppid=360 vsize=390852 CPUtime=2 cores=0,2,4,6
/proc/362/task/373/stat : 373 (ttsth-5-4) R 360 362 330 0 -1 4202560 2784 0 0 0 197 3 0 0 19 0 5 0 516781089 400232448 51164 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 9.05
Current children cumulated vsize (KiB) 390852

[startup+6.30109 s]
/proc/loadavg: 5.37 7.40 7.62 9/186 377
/proc/meminfo: memFree=26643692/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=478832 CPUtime=21.77 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) S 360 362 330 0 -1 4202496 75252 0 0 0 2157 20 0 0 25 0 5 0 516780979 490323968 73508 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 119708 73508 176 36 0 117613 0
[pid=362/tid=370] ppid=360 vsize=478832 CPUtime=5.17 cores=0,2,4,6
/proc/362/task/370/stat : 370 (ttsth-5-4) R 360 362 330 0 -1 4202560 12672 0 0 0 515 2 0 0 25 0 5 0 516781089 490323968 73509 33554432000 4194304 4338100 140733902431984 18446744073709551615 4253966 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=362/tid=371] ppid=360 vsize=478832 CPUtime=5.09 cores=0,2,4,6
/proc/362/task/371/stat : 371 (ttsth-5-4) R 360 362 330 0 -1 4202560 8248 0 0 0 507 2 0 0 18 0 5 0 516781089 490323968 73509 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=362/tid=372] ppid=360 vsize=478832 CPUtime=5.19 cores=0,2,4,6
/proc/362/task/372/stat : 372 (ttsth-5-4) R 360 362 330 0 -1 4202560 11707 0 0 0 518 1 0 0 25 0 5 0 516781089 490323968 73509 33554432000 4194304 4338100 140733902431984 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=362/tid=373] ppid=360 vsize=478832 CPUtime=5.2 cores=0,2,4,6
/proc/362/task/373/stat : 373 (ttsth-5-4) R 360 362 330 0 -1 4202560 6854 0 0 0 510 10 0 0 21 0 5 0 516781089 490323968 73510 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 21.77
Current children cumulated vsize (KiB) 478832

[startup+12.701 s]
/proc/loadavg: 5.78 7.42 7.62 9/186 378
/proc/meminfo: memFree=26349844/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=638888 CPUtime=47.23 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) S 360 362 330 0 -1 4202496 115404 0 0 0 4682 41 0 0 25 0 5 0 516780979 654221312 113660 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 159722 113660 176 36 0 157627 0
[pid=362/tid=370] ppid=360 vsize=638888 CPUtime=11.53 cores=0,2,4,6
/proc/362/task/370/stat : 370 (ttsth-5-4) R 360 362 330 0 -1 4202560 26290 0 0 0 1147 6 0 0 25 0 5 0 516781089 654221312 113660 33554432000 4194304 4338100 140733902431984 18446744073709551615 4206896 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=362/tid=371] ppid=360 vsize=638888 CPUtime=11.38 cores=0,2,4,6
/proc/362/task/371/stat : 371 (ttsth-5-4) R 360 362 330 0 -1 4202560 15898 0 0 0 1133 5 0 0 18 0 5 0 516781089 654221312 113662 33554432000 4194304 4338100 140733902431984 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=362/tid=372] ppid=360 vsize=638888 CPUtime=11.6 cores=0,2,4,6
/proc/362/task/372/stat : 372 (ttsth-5-4) R 360 362 330 0 -1 4202560 23856 0 0 0 1156 4 0 0 25 0 5 0 516781089 654221312 113662 33554432000 4194304 4338100 140733902431984 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=362/tid=373] ppid=360 vsize=638888 CPUtime=11.6 cores=0,2,4,6
/proc/362/task/373/stat : 373 (ttsth-5-4) R 360 362 330 0 -1 4202560 13590 0 0 0 1139 21 0 0 25 0 5 0 516781089 654221312 113663 33554432000 4194304 4338100 140733902431984 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 47.23
Current children cumulated vsize (KiB) 638888

[startup+25.5009 s]
/proc/loadavg: 6.19 7.46 7.63 9/186 378
/proc/meminfo: memFree=25870396/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=902196 CPUtime=98.15 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) S 360 362 330 0 -1 4202496 180331 0 0 0 9739 76 0 0 25 0 5 0 516780979 923848704 178587 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 225549 178587 176 36 0 223454 0
[pid=362/tid=370] ppid=360 vsize=902196 CPUtime=24.26 cores=0,2,4,6
/proc/362/task/370/stat : 370 (ttsth-5-4) R 360 362 330 0 -1 4202560 48273 0 0 0 2413 13 0 0 25 0 5 0 516781089 923848704 178587 33554432000 4194304 4338100 140733902431984 18446744073709551615 210204042665 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=362/tid=371] ppid=360 vsize=902196 CPUtime=23.99 cores=0,2,4,6
/proc/362/task/371/stat : 371 (ttsth-5-4) R 360 362 330 0 -1 4202560 27665 0 0 0 2388 11 0 0 18 0 5 0 516781089 923848704 178587 33554432000 4194304 4338100 140733902431984 18446744073709551615 210204343856 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=362/tid=372] ppid=360 vsize=902196 CPUtime=24.39 cores=0,2,4,6
/proc/362/task/372/stat : 372 (ttsth-5-4) R 360 362 330 0 -1 4202560 43849 0 0 0 2432 7 0 0 25 0 5 0 516781089 923848704 178587 33554432000 4194304 4338100 140733902431984 18446744073709551615 4239484 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=362/tid=373] ppid=360 vsize=902196 CPUtime=24.4 cores=0,2,4,6
/proc/362/task/373/stat : 373 (ttsth-5-4) R 360 362 330 0 -1 4202560 24772 0 0 0 2399 41 0 0 25 0 5 0 516781089 923848704 178587 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 98.15
Current children cumulated vsize (KiB) 902196

[startup+51.1006 s]
/proc/loadavg: 6.90 7.51 7.64 9/186 379
/proc/meminfo: memFree=25171808/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=1298336 CPUtime=200.09 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) S 360 362 330 0 -1 4202496 278461 0 0 0 19872 137 0 0 25 0 5 0 516780979 1329496064 276717 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 324584 276717 176 36 0 322489 0
[pid=362/tid=370] ppid=360 vsize=1298336 CPUtime=49.71 cores=0,2,4,6
/proc/362/task/370/stat : 370 (ttsth-5-4) R 360 362 330 0 -1 4202560 85770 0 0 0 4948 23 0 0 25 0 5 0 516781089 1329496064 276717 33554432000 4194304 4338100 140733902431984 18446744073709551615 4253962 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=362/tid=371] ppid=360 vsize=1298336 CPUtime=49.27 cores=0,2,4,6
/proc/362/task/371/stat : 371 (ttsth-5-4) R 360 362 330 0 -1 4202560 45043 0 0 0 4906 21 0 0 18 0 5 0 516781089 1329496064 276717 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=362/tid=372] ppid=360 vsize=1298336 CPUtime=50 cores=0,2,4,6
/proc/362/task/372/stat : 372 (ttsth-5-4) R 360 362 330 0 -1 4202560 70573 0 0 0 4988 12 0 0 25 0 5 0 516781089 1329496064 276717 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=362/tid=373] ppid=360 vsize=1298336 CPUtime=49.99 cores=0,2,4,6
/proc/362/task/373/stat : 373 (ttsth-5-4) R 360 362 330 0 -1 4202560 41303 0 0 0 4923 76 0 0 25 0 5 0 516781089 1329496064 276717 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 200.09
Current children cumulated vsize (KiB) 1298336

[startup+102.307 s]
/proc/loadavg: 7.52 7.58 7.66 9/186 380
/proc/meminfo: memFree=24055792/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=1895916 CPUtime=404.05 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) S 360 362 330 0 -1 4202496 428770 0 0 0 40174 231 0 0 25 0 5 0 516780979 1941417984 425631 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 473979 425631 176 36 0 471884 0
[pid=362/tid=370] ppid=360 vsize=1895916 CPUtime=100.63 cores=0,2,4,6
/proc/362/task/370/stat : 370 (ttsth-5-4) R 360 362 330 0 -1 4202560 150656 0 0 0 10021 42 0 0 25 0 5 0 516781089 1941417984 425631 33554432000 4194304 4338100 140733902431984 18446744073709551615 4206909 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=362/tid=371] ppid=360 vsize=1895916 CPUtime=99.92 cores=0,2,4,6
/proc/362/task/371/stat : 371 (ttsth-5-4) R 360 362 330 0 -1 4202560 63620 0 0 0 9959 33 0 0 18 0 5 0 516781089 1941417984 425631 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=362/tid=372] ppid=360 vsize=1895916 CPUtime=101.2 cores=0,2,4,6
/proc/362/task/372/stat : 372 (ttsth-5-4) R 360 362 330 0 -1 4202560 123872 0 0 0 10100 20 0 0 25 0 5 0 516781089 1941417984 425631 33554432000 4194304 4338100 140733902431984 18446744073709551615 210204343949 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=362/tid=373] ppid=360 vsize=1895916 CPUtime=101.19 cores=0,2,4,6
/proc/362/task/373/stat : 373 (ttsth-5-4) R 360 362 330 0 -1 4202560 54850 0 0 0 9987 132 0 0 25 0 5 0 516781089 1941417984 425631 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 404.05
Current children cumulated vsize (KiB) 1895916

[startup+162.301 s]
/proc/loadavg: 7.82 7.65 7.67 9/186 382
/proc/meminfo: memFree=23070848/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=2529764 CPUtime=643.1 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) S 360 362 330 0 -1 4202496 587519 0 0 0 63973 337 0 0 25 0 5 0 516780979 2590478336 584380 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 632441 584380 176 36 0 630346 0
[pid=362/tid=370] ppid=360 vsize=2529764 CPUtime=160.55 cores=0,2,4,6
/proc/362/task/370/stat : 370 (ttsth-5-4) R 360 362 330 0 -1 4202560 224583 0 0 0 15990 65 0 0 25 0 5 0 516781089 2590478336 584380 33554432000 4194304 4338100 140733902431984 18446744073709551615 210204042677 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=362/tid=371] ppid=360 vsize=2529764 CPUtime=159.35 cores=0,2,4,6
/proc/362/task/371/stat : 371 (ttsth-5-4) R 360 362 330 0 -1 4202560 79037 0 0 0 15890 45 0 0 18 0 5 0 516781089 2590478336 584380 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=362/tid=372] ppid=360 vsize=2529764 CPUtime=160.9 cores=0,2,4,6
/proc/362/task/372/stat : 372 (ttsth-5-4) R 360 362 330 0 -1 4202560 180942 0 0 0 16059 31 0 0 25 0 5 0 516781089 2590478336 584380 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=362/tid=373] ppid=360 vsize=2529764 CPUtime=161.18 cores=0,2,4,6
/proc/362/task/373/stat : 373 (ttsth-5-4) R 360 362 330 0 -1 4202560 67185 0 0 0 15926 192 0 0 25 0 5 0 516781089 2590478336 584380 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 643.1
Current children cumulated vsize (KiB) 2529764

[startup+222.301 s]
/proc/loadavg: 8.02 7.74 7.70 9/186 383
/proc/meminfo: memFree=22238664/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=3058396 CPUtime=882.18 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) S 360 362 330 0 -1 4202496 719013 0 0 0 87785 433 0 0 25 0 5 0 516780979 3131797504 715874 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 764599 715874 176 36 0 762504 0
[pid=362/tid=370] ppid=360 vsize=3058396 CPUtime=220.53 cores=0,2,4,6
/proc/362/task/370/stat : 370 (ttsth-5-4) R 360 362 330 0 -1 4202560 295631 0 0 0 21970 83 0 0 25 0 5 0 516781089 3131797504 715874 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0

################
# More data... #
################

/proc/362/task/373/stat : 373 (ttsth-5-4) R 360 362 330 0 -1 4202560 95355 0 0 0 69214 887 0 0 25 0 5 0 516781089 4463599616 1021514 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2796.92
Current children cumulated vsize (KiB) 4358984

[startup+762.307 s]
/proc/loadavg: 8.07 8.02 7.85 9/186 398
/proc/meminfo: memFree=18799844/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=4427932 CPUtime=3036.54 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) S 360 362 330 0 -1 4202496 1042067 0 0 0 301820 1834 0 0 25 0 5 0 516780979 4534202368 1038928 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 1106983 1038928 176 36 0 1104888 0
[pid=362/tid=370] ppid=360 vsize=4427932 CPUtime=759.08 cores=0,2,4,6
/proc/362/task/370/stat : 370 (ttsth-5-4) R 360 362 330 0 -1 4202560 500297 0 0 0 75736 172 0 0 25 0 5 0 516781089 4534202368 1038928 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=362/tid=371] ppid=360 vsize=4427932 CPUtime=756.5 cores=0,2,4,6
/proc/362/task/371/stat : 371 (ttsth-5-4) R 360 362 330 0 -1 4202560 104320 0 0 0 75048 602 0 0 25 0 5 0 516781089 4534202368 1038928 33554432000 4194304 4338100 140733902431984 18446744073709551615 4253933 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=362/tid=372] ppid=360 vsize=4427932 CPUtime=759.01 cores=0,2,4,6
/proc/362/task/372/stat : 372 (ttsth-5-4) R 360 362 330 0 -1 4202560 306321 0 0 0 75842 59 0 0 25 0 5 0 516781089 4534202368 1038928 33554432000 4194304 4338100 140733902431984 18446744073709551615 210204043784 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=362/tid=373] ppid=360 vsize=4427932 CPUtime=760.82 cores=0,2,4,6
/proc/362/task/373/stat : 373 (ttsth-5-4) R 360 362 330 0 -1 4202560 95357 0 0 0 75086 996 0 0 25 0 5 0 516781089 4534202368 1038928 33554432000 4194304 4338100 140733902431984 18446744073709551615 210204043002 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3036.54
Current children cumulated vsize (KiB) 4427932

[startup+822.301 s]
/proc/loadavg: 8.08 8.03 7.87 9/186 399
/proc/meminfo: memFree=18509168/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=4562436 CPUtime=3276.15 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) S 360 362 330 0 -1 4202496 1076166 0 0 0 325562 2053 0 0 25 0 5 0 516780979 4671934464 1073027 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 1140609 1073027 176 36 0 1138514 0
[pid=362/tid=370] ppid=360 vsize=4562436 CPUtime=819.08 cores=0,2,4,6
/proc/362/task/370/stat : 370 (ttsth-5-4) R 360 362 330 0 -1 4202560 525843 0 0 0 81727 181 0 0 25 0 5 0 516781089 4671934464 1073027 33554432000 4194304 4338100 140733902431984 18446744073709551615 4279533 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=362/tid=371] ppid=360 vsize=4562436 CPUtime=816.48 cores=0,2,4,6
/proc/362/task/371/stat : 371 (ttsth-5-4) R 360 362 330 0 -1 4202560 104321 0 0 0 80940 708 0 0 25 0 5 0 516781089 4671934464 1073027 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254384 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=362/tid=372] ppid=360 vsize=4562436 CPUtime=819 cores=0,2,4,6
/proc/362/task/372/stat : 372 (ttsth-5-4) R 360 362 330 0 -1 4202560 314870 0 0 0 81839 61 0 0 25 0 5 0 516781089 4671934464 1073027 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=362/tid=373] ppid=360 vsize=4562436 CPUtime=820.48 cores=0,2,4,6
/proc/362/task/373/stat : 373 (ttsth-5-4) R 360 362 330 0 -1 4202560 95360 0 0 0 80949 1099 0 0 25 0 5 0 516781089 4671934464 1073027 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3276.15
Current children cumulated vsize (KiB) 4562436

[startup+882.301 s]
/proc/loadavg: 8.07 8.04 7.88 9/186 401
/proc/meminfo: memFree=18288812/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=4571820 CPUtime=3515.79 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) S 360 362 330 0 -1 4202496 1077976 0 0 0 349338 2241 0 0 25 0 5 0 516780979 4681543680 1074837 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 1142955 1074837 176 36 0 1140860 0
[pid=362/tid=370] ppid=360 vsize=4571820 CPUtime=879.05 cores=0,2,4,6
/proc/362/task/370/stat : 370 (ttsth-5-4) R 360 362 330 0 -1 4202560 527647 0 0 0 87721 184 0 0 25 0 5 0 516781089 4681543680 1074837 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=362/tid=371] ppid=360 vsize=4571820 CPUtime=876.31 cores=0,2,4,6
/proc/362/task/371/stat : 371 (ttsth-5-4) R 360 362 330 0 -1 4202560 104322 0 0 0 86830 801 0 0 25 0 5 0 516781089 4681543680 1074837 33554432000 4194304 4338100 140733902431984 18446744073709551615 4284164 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=362/tid=372] ppid=360 vsize=4571820 CPUtime=878.99 cores=0,2,4,6
/proc/362/task/372/stat : 372 (ttsth-5-4) R 360 362 330 0 -1 4202560 314875 0 0 0 87838 61 0 0 25 0 5 0 516781089 4681543680 1074837 33554432000 4194304 4338100 140733902431984 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=362/tid=373] ppid=360 vsize=4571820 CPUtime=880.31 cores=0,2,4,6
/proc/362/task/373/stat : 373 (ttsth-5-4) R 360 362 330 0 -1 4202560 95360 0 0 0 86841 1190 0 0 25 0 5 0 516781089 4681543680 1074837 33554432000 4194304 4338100 140733902431984 18446744073709551615 4284123 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3515.79
Current children cumulated vsize (KiB) 4571820

[startup+942.301 s]
/proc/loadavg: 8.11 8.06 7.89 9/186 403
/proc/meminfo: memFree=17918144/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=4737604 CPUtime=3755.45 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) S 360 362 330 0 -1 4202496 1119799 0 0 0 373106 2439 0 0 25 0 5 0 516780979 4851306496 1116660 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 1184401 1116660 176 36 0 1182306 0
[pid=362/tid=370] ppid=360 vsize=4737604 CPUtime=939.05 cores=0,2,4,6
/proc/362/task/370/stat : 370 (ttsth-5-4) R 360 362 330 0 -1 4202560 551132 0 0 0 93711 194 0 0 25 0 5 0 516781089 4851306496 1116660 33554432000 4194304 4338100 140733902431984 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=362/tid=371] ppid=360 vsize=4737604 CPUtime=936.01 cores=0,2,4,6
/proc/362/task/371/stat : 371 (ttsth-5-4) R 360 362 330 0 -1 4202560 104322 0 0 0 92707 894 0 0 25 0 5 0 516781089 4851306496 1116660 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=362/tid=372] ppid=360 vsize=4737604 CPUtime=938.99 cores=0,2,4,6
/proc/362/task/372/stat : 372 (ttsth-5-4) R 360 362 330 0 -1 4202560 333212 0 0 0 93834 65 0 0 25 0 5 0 516781089 4851306496 1116660 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=362/tid=373] ppid=360 vsize=4737604 CPUtime=940.29 cores=0,2,4,6
/proc/362/task/373/stat : 373 (ttsth-5-4) R 360 362 330 0 -1 4202560 95361 0 0 0 92747 1282 0 0 25 0 5 0 516781089 4851306496 1116660 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3755.45
Current children cumulated vsize (KiB) 4737604

[startup+1002.3 s]
/proc/loadavg: 8.12 8.07 7.91 9/186 404
/proc/meminfo: memFree=17523776/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=5037892 CPUtime=3995.1 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) S 360 362 330 0 -1 4202496 1194493 0 0 0 396838 2672 0 0 25 0 5 0 516780979 5158801408 1191354 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 1259473 1191354 176 36 0 1257378 0
[pid=362/tid=370] ppid=360 vsize=5037892 CPUtime=999.04 cores=0,2,4,6
/proc/362/task/370/stat : 370 (ttsth-5-4) R 360 362 330 0 -1 4202560 587817 0 0 0 99698 206 0 0 25 0 5 0 516781089 5158801408 1191354 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254041 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=362/tid=371] ppid=360 vsize=5037892 CPUtime=995.99 cores=0,2,4,6
/proc/362/task/371/stat : 371 (ttsth-5-4) R 360 362 330 0 -1 4202560 104324 0 0 0 98600 999 0 0 25 0 5 0 516781089 5158801408 1191354 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=362/tid=372] ppid=360 vsize=5037892 CPUtime=998.97 cores=0,2,4,6
/proc/362/task/372/stat : 372 (ttsth-5-4) R 360 362 330 0 -1 4202560 371216 0 0 0 99821 76 0 0 25 0 5 0 516781089 5158801408 1191354 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254462 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=362/tid=373] ppid=360 vsize=5037892 CPUtime=999.95 cores=0,2,4,6
/proc/362/task/373/stat : 373 (ttsth-5-4) R 360 362 330 0 -1 4202560 95364 0 0 0 98610 1385 0 0 25 0 5 0 516781089 5158801408 1191354 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3995.1
Current children cumulated vsize (KiB) 5037892

[startup+1062.3 s]
/proc/loadavg: 8.08 8.07 7.91 9/186 406
/proc/meminfo: memFree=17120508/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=5366332 CPUtime=4234.68 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) S 360 362 330 0 -1 4202496 1277677 0 0 0 420577 2891 0 0 25 0 5 0 516780979 5495123968 1274538 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 1341583 1274538 176 36 0 1339488 0
[pid=362/tid=370] ppid=360 vsize=5366332 CPUtime=1059.01 cores=0,2,4,6
/proc/362/task/370/stat : 370 (ttsth-5-4) R 360 362 330 0 -1 4202560 623837 0 0 0 105684 217 0 0 18 0 5 0 516781089 5495123968 1274538 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=362/tid=371] ppid=360 vsize=5366332 CPUtime=1055.98 cores=0,2,4,6
/proc/362/task/371/stat : 371 (ttsth-5-4) R 360 362 330 0 -1 4202560 111803 0 0 0 104497 1101 0 0 25 0 5 0 516781089 5495123968 1274538 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=362/tid=372] ppid=360 vsize=5366332 CPUtime=1058.96 cores=0,2,4,6
/proc/362/task/372/stat : 372 (ttsth-5-4) R 360 362 330 0 -1 4202560 402052 0 0 0 105812 84 0 0 25 0 5 0 516781089 5495123968 1274538 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254362 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=362/tid=373] ppid=360 vsize=5366332 CPUtime=1059.61 cores=0,2,4,6
/proc/362/task/373/stat : 373 (ttsth-5-4) R 360 362 330 0 -1 4202560 104213 0 0 0 104477 1484 0 0 25 0 5 0 516781089 5495123968 1274538 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4234.68
Current children cumulated vsize (KiB) 5366332

[startup+1122.3 s]
/proc/loadavg: 8.06 8.07 7.92 9/186 407
/proc/meminfo: memFree=16765728/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=5554144 CPUtime=4474.32 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) S 360 362 330 0 -1 4202496 1323632 0 0 0 444336 3096 0 0 25 0 5 0 516780979 5687443456 1320493 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 1388536 1320493 176 36 0 1386441 0
[pid=362/tid=370] ppid=360 vsize=5554144 CPUtime=1119.01 cores=0,2,4,6
/proc/362/task/370/stat : 370 (ttsth-5-4) R 360 362 330 0 -1 4202560 644269 0 0 0 111674 227 0 0 25 0 5 0 516781089 5687443456 1320493 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=362/tid=371] ppid=360 vsize=5554144 CPUtime=1115.99 cores=0,2,4,6
/proc/362/task/371/stat : 371 (ttsth-5-4) R 360 362 330 0 -1 4202560 116908 0 0 0 110403 1196 0 0 25 0 5 0 516781089 5687443456 1320493 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=362/tid=372] ppid=360 vsize=5554144 CPUtime=1118.94 cores=0,2,4,6
/proc/362/task/372/stat : 372 (ttsth-5-4) R 360 362 330 0 -1 4202560 418642 0 0 0 111807 87 0 0 25 0 5 0 516781089 5687443456 1320493 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=362/tid=373] ppid=360 vsize=5554144 CPUtime=1119.27 cores=0,2,4,6
/proc/362/task/373/stat : 373 (ttsth-5-4) R 360 362 330 0 -1 4202560 108041 0 0 0 110345 1582 0 0 25 0 5 0 516781089 5687443456 1320493 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4474.32
Current children cumulated vsize (KiB) 5554144

[startup+1182.3 s]
/proc/loadavg: 8.13 8.09 7.93 9/186 409
/proc/meminfo: memFree=16462144/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=5691776 CPUtime=4713.99 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) S 360 362 330 0 -1 4202496 1358294 0 0 0 468101 3298 0 0 25 0 5 0 516780979 5828378624 1355155 33554432000 4194304 4338100 140733902431984 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 1422944 1355155 176 36 0 1420849 0
[pid=362/tid=370] ppid=360 vsize=5691776 CPUtime=1179 cores=0,2,4,6
/proc/362/task/370/stat : 370 (ttsth-5-4) R 360 362 330 0 -1 4202560 672722 0 0 0 117663 237 0 0 25 0 5 0 516781089 5828378624 1355155 33554432000 4194304 4338100 140733902431984 18446744073709551615 210204042470 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=362/tid=371] ppid=360 vsize=5691776 CPUtime=1175.98 cores=0,2,4,6
/proc/362/task/371/stat : 371 (ttsth-5-4) R 360 362 330 0 -1 4202560 116913 0 0 0 116309 1289 0 0 25 0 5 0 516781089 5828378624 1355155 33554432000 4194304 4338100 140733902431984 18446744073709551615 210204343838 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=362/tid=372] ppid=360 vsize=5691776 CPUtime=1178.94 cores=0,2,4,6
/proc/362/task/372/stat : 372 (ttsth-5-4) R 360 362 330 0 -1 4202560 424844 0 0 0 117805 89 0 0 25 0 5 0 516781089 5828378624 1355155 33554432000 4194304 4338100 140733902431984 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=362/tid=373] ppid=360 vsize=5691776 CPUtime=1178.94 cores=0,2,4,6
/proc/362/task/373/stat : 373 (ttsth-5-4) R 360 362 330 0 -1 4202560 108043 0 0 0 116216 1678 0 0 25 0 5 0 516781089 5828378624 1355155 33554432000 4194304 4338100 140733902431984 18446744073709551615 4206915 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4713.99
Current children cumulated vsize (KiB) 5691776



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.9 s]
/proc/loadavg: 8.09 8.08 7.93 2/177 410
/proc/meminfo: memFree=22861944/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=0 CPUtime=4800.05 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) Z 360 362 330 0 -1 4203548 1358297 0 0 0 476634 3371 0 0 18 0 2 0 516780979 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 0 0 0 0 0 0 0
[pid=362/tid=371] ppid=360 vsize=0 CPUtime=1197.57 cores=0,2,4,6
/proc/362/task/371/stat : 371 (ttsth-5-4) R 360 362 330 0 -1 4203588 116913 0 0 0 118429 1328 0 0 25 0 2 0 516781089 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.05
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1204.01 s]
/proc/loadavg: 8.09 8.08 7.93 2/177 410
/proc/meminfo: memFree=22861944/32951124 swapFree=67088964/67111528
[pid=362] ppid=360 vsize=0 CPUtime=4800.16 cores=0,2,4,6
/proc/362/stat : 362 (ttsth-5-4) Z 360 362 330 0 -1 4203548 1358297 0 0 0 476634 3382 0 0 18 0 2 0 516780979 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/362/statm: 0 0 0 0 0 0 0
[pid=362/tid=371] ppid=360 vsize=0 CPUtime=1197.68 cores=0,2,4,6
/proc/362/task/371/stat : 371 (ttsth-5-4) R 360 362 330 0 -1 4203588 116913 0 0 0 118429 1339 0 0 25 0 2 0 516781089 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.16
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1204.07
CPU time (s): 4800.23
CPU user time (s): 4766.35
CPU system time (s): 33.8818
CPU usage (%): 398.668
Max. virtual memory (cumulated for all children) (KiB): 5691776

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4766.35
system time used= 33.8818
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1358297
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 8772
involuntary context switches= 22705

runsolver used 1.90171 second user time and 4.66829 second system time

The end

Launcher Data 

Begin job on node130 at 2011-04-29 06:37:05
IDJOB=3258526
IDBENCH=71367
IDSOLVER=1654
FILE ID=node130/3258526-1304051825
RUNJOBID= node130-1304051825-348
PBS_JOBID= 13170415
Free space on /tmp= 71988 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT09/CRAFTED/rbsat/random/forced/rbsat-v2100c215164gyes7.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3258526-1304051825/watcher-3258526-1304051825 -o /tmp/evaluation-result-3258526-1304051825/solver-3258526-1304051825 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3258526-1304051825.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 140eea74f7e66501965365160d02fa4d
RANDOM SEED=1889897766

node130.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      27177160 kB
Buffers:       1155600 kB
Cached:        4095568 kB
SwapCached:      22432 kB
Active:         357732 kB
Inactive:      4983124 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      27177160 kB
SwapTotal:    67111528 kB
SwapFree:     67088964 kB
Dirty:            5552 kB
Writeback:           0 kB
AnonPages:       66760 kB
Mapped:          14844 kB
Slab:           368304 kB
PageTables:       4988 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   192844 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71988 MiB
End job on node130 at 2011-04-29 06:57:11