Trace number 3370900

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.22 1203.67

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m42-q663-n40-p15-s72490337.cnf
MD5SUMab8e40d31c79e7a71114e3126a46874f
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark1.38479
Satisfiable
(Un)Satisfiability was proved
Number of variables1649
Number of clauses256038
Sum of the clauses size513645
Maximum clause length42
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2255998
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 540

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3370900-1305433137/watcher-3370900-1305433137 -o /tmp/evaluation-result-3370900-1305433137/solver-3370900-1305433137 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3370900-1305433137.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.67 7.77 7.94 2/354 22030
/proc/meminfo: memFree=21253296/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=9668 CPUtime=0 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) R 22028 22030 21414 0 -1 4202496 509 1259 0 0 0 0 0 0 23 0 1 0 654911444 9900032 439 33554432000 4194304 4417452 140736591993600 18446744073709551615 218905730816 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/22030/statm: 2417 439 147 55 0 303 0

[startup+0.0512219 s]
/proc/loadavg: 6.67 7.77 7.94 2/354 22030
/proc/meminfo: memFree=21253296/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=17684 CPUtime=0.04 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) R 22028 22030 21414 0 -1 4202496 2562 1259 0 0 4 0 0 0 24 0 1 0 654911444 18108416 2444 33554432000 4194304 4417452 140736591993600 18446744073709551615 4209666 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/22030/statm: 4421 2444 148 55 0 2307 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 17684

[startup+0.100227 s]
/proc/loadavg: 6.67 7.77 7.94 2/354 22030
/proc/meminfo: memFree=21253296/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=25876 CPUtime=0.09 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) R 22028 22030 21414 0 -1 4202496 4610 1259 0 0 9 0 0 0 24 0 1 0 654911444 26497024 4492 33554432000 4194304 4417452 140736591993600 18446744073709551615 4235968 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/22030/statm: 6469 4492 148 55 0 4355 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 25876

[startup+0.300144 s]
/proc/loadavg: 6.67 7.77 7.94 2/354 22030
/proc/meminfo: memFree=21253296/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=85940 CPUtime=0.68 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 9673 1259 0 0 67 1 0 0 24 0 5 0 654911444 88002560 9254 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 21485 9254 175 55 0 19371 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 85940

[startup+0.70107 s]
/proc/loadavg: 6.67 7.77 7.94 2/354 22030
/proc/meminfo: memFree=21253296/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=165392 CPUtime=2.29 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 20598 1259 0 0 226 3 0 0 24 0 5 0 654911444 169361408 12772 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 41348 12772 179 55 0 39234 0
Current children cumulated CPU time (s) 2.29
Current children cumulated vsize (KiB) 165392

[startup+1.50093 s]
/proc/loadavg: 6.67 7.77 7.94 10/363 22044
/proc/meminfo: memFree=21154704/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=168712 CPUtime=5.47 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 25315 1259 0 0 544 3 0 0 24 0 5 0 654911444 172761088 13602 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 42178 13602 179 55 0 40064 0
[pid=22030/tid=22041] ppid=22028 vsize=168712 CPUtime=1.32 cores=1,3,5,7
/proc/22030/task/22041/stat : 22041 (plingeling) R 22028 22030 21414 0 -1 4202560 2765 1259 0 0 132 0 0 0 25 0 5 0 654911461 172761088 13602 33554432000 4194304 4417452 140736591993600 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22030/tid=22042] ppid=22028 vsize=168712 CPUtime=1.32 cores=1,3,5,7
/proc/22030/task/22042/stat : 22042 (plingeling) R 22028 22030 21414 0 -1 4202560 3048 1259 0 0 132 0 0 0 25 0 5 0 654911461 172761088 13602 33554432000 4194304 4417452 140736591993600 18446744073709551615 4258161 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22030/tid=22043] ppid=22028 vsize=168712 CPUtime=1.33 cores=1,3,5,7
/proc/22030/task/22043/stat : 22043 (plingeling) R 22028 22030 21414 0 -1 4202560 2397 1259 0 0 133 0 0 0 25 0 5 0 654911461 172761088 13602 33554432000 4194304 4417452 140736591993600 18446744073709551615 4376689 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22030/tid=22044] ppid=22028 vsize=168712 CPUtime=1.31 cores=1,3,5,7
/proc/22030/task/22044/stat : 22044 (plingeling) R 22028 22030 21414 0 -1 4202560 8387 1259 0 0 130 1 0 0 25 0 5 0 654911461 172761088 13602 33554432000 4194304 4417452 140736591993600 18446744073709551615 4365111 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.47
Current children cumulated vsize (KiB) 168712

[startup+3.10064 s]
/proc/loadavg: 6.67 7.77 7.94 10/363 22044
/proc/meminfo: memFree=21132508/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=181328 CPUtime=11.86 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 31494 1259 0 0 1182 4 0 0 24 0 5 0 654911444 185679872 16756 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 45332 16756 179 55 0 43218 0
[pid=22030/tid=22041] ppid=22028 vsize=181328 CPUtime=2.92 cores=1,3,5,7
/proc/22030/task/22041/stat : 22041 (plingeling) R 22028 22030 21414 0 -1 4202560 2765 1259 0 0 292 0 0 0 25 0 5 0 654911461 185679872 16756 33554432000 4194304 4417452 140736591993600 18446744073709551615 4249991 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22030/tid=22042] ppid=22028 vsize=181328 CPUtime=2.92 cores=1,3,5,7
/proc/22030/task/22042/stat : 22042 (plingeling) R 22028 22030 21414 0 -1 4202560 3048 1259 0 0 292 0 0 0 25 0 5 0 654911461 185679872 16756 33554432000 4194304 4417452 140736591993600 18446744073709551615 4250022 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22030/tid=22043] ppid=22028 vsize=181328 CPUtime=2.93 cores=1,3,5,7
/proc/22030/task/22043/stat : 22043 (plingeling) R 22028 22030 21414 0 -1 4202560 2397 1259 0 0 293 0 0 0 25 0 5 0 654911461 185679872 16756 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364896 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22030/tid=22044] ppid=22028 vsize=181328 CPUtime=2.9 cores=1,3,5,7
/proc/22030/task/22044/stat : 22044 (plingeling) R 22028 22030 21414 0 -1 4202560 14566 1259 0 0 288 2 0 0 25 0 5 0 654911461 185679872 16756 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.86
Current children cumulated vsize (KiB) 181328

[startup+6.30106 s]
/proc/loadavg: 6.77 7.78 7.94 10/363 22044
/proc/meminfo: memFree=21098156/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=199228 CPUtime=24.63 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 41530 1259 0 0 2457 6 0 0 24 0 5 0 654911444 204009472 20947 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 49807 20947 179 55 0 47693 0
[pid=22030/tid=22041] ppid=22028 vsize=199228 CPUtime=6.12 cores=1,3,5,7
/proc/22030/task/22041/stat : 22041 (plingeling) R 22028 22030 21414 0 -1 4202560 2765 1259 0 0 612 0 0 0 25 0 5 0 654911461 204009472 20947 33554432000 4194304 4417452 140736591993600 18446744073709551615 4250072 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22030/tid=22042] ppid=22028 vsize=199228 CPUtime=6.12 cores=1,3,5,7
/proc/22030/task/22042/stat : 22042 (plingeling) R 22028 22030 21414 0 -1 4202560 3048 1259 0 0 612 0 0 0 25 0 5 0 654911461 204009472 20947 33554432000 4194304 4417452 140736591993600 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22030/tid=22043] ppid=22028 vsize=199228 CPUtime=6.12 cores=1,3,5,7
/proc/22030/task/22043/stat : 22043 (plingeling) R 22028 22030 21414 0 -1 4202560 4223 1259 0 0 612 0 0 0 25 0 5 0 654911461 204009472 20947 33554432000 4194304 4417452 140736591993600 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22030/tid=22044] ppid=22028 vsize=199228 CPUtime=6.06 cores=1,3,5,7
/proc/22030/task/22044/stat : 22044 (plingeling) R 22028 22030 21414 0 -1 4202560 22776 1259 0 0 603 3 0 0 25 0 5 0 654911461 204009472 20947 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.63
Current children cumulated vsize (KiB) 199228

[startup+12.7009 s]
/proc/loadavg: 6.95 7.80 7.95 10/363 22045
/proc/meminfo: memFree=20963960/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=271736 CPUtime=50.15 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 74039 1259 0 0 5004 11 0 0 24 0 5 0 654911444 278257664 39358 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 67934 39358 179 55 0 65820 0
[pid=22030/tid=22041] ppid=22028 vsize=271736 CPUtime=12.52 cores=1,3,5,7
/proc/22030/task/22041/stat : 22041 (plingeling) R 22028 22030 21414 0 -1 4202560 2765 1259 0 0 1252 0 0 0 25 0 5 0 654911461 278257664 39358 33554432000 4194304 4417452 140736591993600 18446744073709551615 4250017 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22030/tid=22042] ppid=22028 vsize=271736 CPUtime=12.52 cores=1,3,5,7
/proc/22030/task/22042/stat : 22042 (plingeling) R 22028 22030 21414 0 -1 4202560 3048 1259 0 0 1252 0 0 0 25 0 5 0 654911461 278257664 39358 33554432000 4194304 4417452 140736591993600 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22030/tid=22043] ppid=22028 vsize=271736 CPUtime=12.52 cores=1,3,5,7
/proc/22030/task/22043/stat : 22043 (plingeling) R 22028 22030 21414 0 -1 4202560 4795 1259 0 0 1252 0 0 0 25 0 5 0 654911461 278257664 39358 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22030/tid=22044] ppid=22028 vsize=271736 CPUtime=12.39 cores=1,3,5,7
/proc/22030/task/22044/stat : 22044 (plingeling) R 22028 22030 21414 0 -1 4202560 54713 1259 0 0 1230 9 0 0 25 0 5 0 654911461 278257664 39358 33554432000 4194304 4417452 140736591993600 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.15
Current children cumulated vsize (KiB) 271736

[startup+25.5006 s]
/proc/loadavg: 7.18 7.81 7.95 9/363 22045
/proc/meminfo: memFree=20905200/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=310928 CPUtime=101.21 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 105267 1259 0 0 10105 16 0 0 24 0 5 0 654911444 318390272 49157 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 77732 49157 180 55 0 75618 0
[pid=22030/tid=22041] ppid=22028 vsize=310928 CPUtime=25.32 cores=1,3,5,7
/proc/22030/task/22041/stat : 22041 (plingeling) R 22028 22030 21414 0 -1 4202560 6256 1259 0 0 2532 0 0 0 25 0 5 0 654911461 318390272 49157 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22030/tid=22042] ppid=22028 vsize=310928 CPUtime=25.32 cores=1,3,5,7
/proc/22030/task/22042/stat : 22042 (plingeling) R 22028 22030 21414 0 -1 4202560 4844 1259 0 0 2532 0 0 0 25 0 5 0 654911461 318390272 49157 33554432000 4194304 4417452 140736591993600 18446744073709551615 4366469 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22030/tid=22043] ppid=22028 vsize=310928 CPUtime=25.32 cores=1,3,5,7
/proc/22030/task/22043/stat : 22043 (plingeling) R 22028 22030 21414 0 -1 4202560 8893 1259 0 0 2532 0 0 0 25 0 5 0 654911461 318390272 49157 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22030/tid=22044] ppid=22028 vsize=310928 CPUtime=25.05 cores=1,3,5,7
/proc/22030/task/22044/stat : 22044 (plingeling) R 22028 22030 21414 0 -1 4202560 76556 1259 0 0 2492 13 0 0 25 0 5 0 654911461 318390272 49157 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.21
Current children cumulated vsize (KiB) 310928

[startup+51.101 s]
/proc/loadavg: 7.46 7.82 7.95 10/363 22046
/proc/meminfo: memFree=20707104/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=415084 CPUtime=203.34 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 153403 1259 0 0 20310 24 0 0 24 0 5 0 654911444 425046016 75196 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 103771 75196 180 55 0 101657 0
[pid=22030/tid=22041] ppid=22028 vsize=415084 CPUtime=50.92 cores=1,3,5,7
/proc/22030/task/22041/stat : 22041 (plingeling) R 22028 22030 21414 0 -1 4202560 11248 1259 0 0 5091 1 0 0 25 0 5 0 654911461 425046016 75196 33554432000 4194304 4417452 140736591993600 18446744073709551615 4258267 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22030/tid=22042] ppid=22028 vsize=415084 CPUtime=50.92 cores=1,3,5,7
/proc/22030/task/22042/stat : 22042 (plingeling) R 22028 22030 21414 0 -1 4202560 5334 1259 0 0 5092 0 0 0 25 0 5 0 654911461 425046016 75196 33554432000 4194304 4417452 140736591993600 18446744073709551615 4250022 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22030/tid=22043] ppid=22028 vsize=415084 CPUtime=50.93 cores=1,3,5,7
/proc/22030/task/22043/stat : 22043 (plingeling) R 22028 22030 21414 0 -1 4202560 11239 1259 0 0 5092 1 0 0 25 0 5 0 654911461 425046016 75196 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22030/tid=22044] ppid=22028 vsize=415084 CPUtime=50.37 cores=1,3,5,7
/proc/22030/task/22044/stat : 22044 (plingeling) R 22028 22030 21414 0 -1 4202560 116864 1259 0 0 5018 19 0 0 25 0 5 0 654911461 425046016 75196 33554432000 4194304 4417452 140736591993600 18446744073709551615 4358947 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.34
Current children cumulated vsize (KiB) 415084

[startup+102.314 s]
/proc/loadavg: 7.81 7.86 7.95 10/363 22047
/proc/meminfo: memFree=20392492/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=509284 CPUtime=407.62 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 304491 1259 0 0 40715 47 0 0 24 0 5 0 654911444 521506816 98746 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 127321 98746 180 55 0 125207 0
[pid=22030/tid=22041] ppid=22028 vsize=509284 CPUtime=102.14 cores=1,3,5,7
/proc/22030/task/22041/stat : 22041 (plingeling) R 22028 22030 21414 0 -1 4202560 43632 1259 0 0 10208 6 0 0 25 0 5 0 654911461 521506816 98746 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22030/tid=22042] ppid=22028 vsize=509284 CPUtime=102.13 cores=1,3,5,7
/proc/22030/task/22042/stat : 22042 (plingeling) R 22028 22030 21414 0 -1 4202560 26323 1259 0 0 10210 3 0 0 25 0 5 0 654911461 521506816 98746 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22030/tid=22043] ppid=22028 vsize=509284 CPUtime=102.13 cores=1,3,5,7
/proc/22030/task/22043/stat : 22043 (plingeling) R 22028 22030 21414 0 -1 4202560 33528 1259 0 0 10210 3 0 0 25 0 5 0 654911461 521506816 98746 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22030/tid=22044] ppid=22028 vsize=509284 CPUtime=101.01 cores=1,3,5,7
/proc/22030/task/22044/stat : 22044 (plingeling) R 22028 22030 21414 0 -1 4202560 192290 1259 0 0 10069 32 0 0 25 0 5 0 654911461 521506816 98746 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 407.62
Current children cumulated vsize (KiB) 509284

[startup+162.301 s]
/proc/loadavg: 8.01 7.91 7.96 9/363 22049
/proc/meminfo: memFree=19991448/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=823388 CPUtime=646.91 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 439794 1259 0 0 64624 67 0 0 24 0 5 0 654911444 843149312 177272 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 205847 177272 180 55 0 203733 0
[pid=22030/tid=22041] ppid=22028 vsize=823388 CPUtime=162.13 cores=1,3,5,7
/proc/22030/task/22041/stat : 22041 (plingeling) R 22028 22030 21414 0 -1 4202560 74284 1259 0 0 16203 10 0 0 25 0 5 0 654911461 843149312 177272 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22030/tid=22042] ppid=22028 vsize=823388 CPUtime=162.13 cores=1,3,5,7
/proc/22030/task/22042/stat : 22042 (plingeling) R 22028 22030 21414 0 -1 4202560 61794 1259 0 0 16205 8 0 0 25 0 5 0 654911461 843149312 177272 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22030/tid=22043] ppid=22028 vsize=823388 CPUtime=162.12 cores=1,3,5,7
/proc/22030/task/22043/stat : 22043 (plingeling) R 22028 22030 21414 0 -1 4202560 61953 1259 0 0 16205 7 0 0 25 0 5 0 654911461 843149312 177272 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22030/tid=22044] ppid=22028 vsize=823388 CPUtime=160.35 cores=1,3,5,7
/proc/22030/task/22044/stat : 22044 (plingeling) R 22028 22030 21414 0 -1 4202560 233045 1259 0 0 15995 40 0 0 25 0 5 0 654911461 843149312 177272 33554432000 4194304 4417452 140736591993600 18446744073709551615 4366385 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 646.91
Current children cumulated vsize (KiB) 823388


################
# More data... #
################

[pid=22030/tid=22042] ppid=22028 vsize=2019484 CPUtime=702.14 cores=1,3,5,7
/proc/22030/task/22042/stat : 22042 (plingeling) R 22028 22030 21414 0 -1 4202560 383726 1259 0 0 70165 49 0 0 25 0 5 0 654911461 2067951616 476296 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22030/tid=22043] ppid=22028 vsize=2019484 CPUtime=702.14 cores=1,3,5,7
/proc/22030/task/22043/stat : 22043 (plingeling) R 22028 22030 21414 0 -1 4202560 486855 1259 0 0 70155 59 0 0 25 0 5 0 654911461 2067951616 476296 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22030/tid=22044] ppid=22028 vsize=2019484 CPUtime=694.12 cores=1,3,5,7
/proc/22030/task/22044/stat : 22044 (plingeling) R 22028 22030 21414 0 -1 4202560 527663 1259 0 0 69321 91 0 0 25 0 5 0 654911461 2067951616 476296 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2800.73
Current children cumulated vsize (KiB) 2019484

[startup+762.313 s]
/proc/loadavg: 8.17 8.11 8.02 10/363 22065
/proc/meminfo: memFree=18106920/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=1808140 CPUtime=3040.08 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 1935319 1259 0 0 303734 274 0 0 24 0 5 0 654911444 1851535360 423492 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 452035 423492 180 55 0 449921 0
[pid=22030/tid=22041] ppid=22028 vsize=1808140 CPUtime=762.16 cores=1,3,5,7
/proc/22030/task/22041/stat : 22041 (plingeling) R 22028 22030 21414 0 -1 4202560 436618 1259 0 0 76155 61 0 0 25 0 5 0 654911461 1851535360 423492 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22030/tid=22042] ppid=22028 vsize=1808140 CPUtime=762.15 cores=1,3,5,7
/proc/22030/task/22042/stat : 22042 (plingeling) R 22028 22030 21414 0 -1 4202560 426313 1259 0 0 76160 55 0 0 25 0 5 0 654911461 1851535360 423492 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22030/tid=22043] ppid=22028 vsize=1808140 CPUtime=762.16 cores=1,3,5,7
/proc/22030/task/22043/stat : 22043 (plingeling) R 22028 22030 21414 0 -1 4202560 536007 1259 0 0 76151 65 0 0 25 0 5 0 654911461 1851535360 423492 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22030/tid=22044] ppid=22028 vsize=1808140 CPUtime=753.42 cores=1,3,5,7
/proc/22030/task/22044/stat : 22044 (plingeling) R 22028 22030 21414 0 -1 4202560 527663 1259 0 0 75251 91 0 0 25 0 5 0 654911461 1851535360 423492 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3040.08
Current children cumulated vsize (KiB) 1808140

[startup+822.3 s]
/proc/loadavg: 8.17 8.11 8.03 9/363 22066
/proc/meminfo: memFree=17813368/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=1899616 CPUtime=3279.33 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 1999148 1259 0 0 327650 283 0 0 24 0 5 0 654911444 1945206784 446361 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 474904 446361 180 55 0 472790 0
[pid=22030/tid=22041] ppid=22028 vsize=1899616 CPUtime=822.15 cores=1,3,5,7
/proc/22030/task/22041/stat : 22041 (plingeling) R 22028 22030 21414 0 -1 4202560 444810 1259 0 0 82153 62 0 0 25 0 5 0 654911461 1945206784 446361 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22030/tid=22042] ppid=22028 vsize=1899616 CPUtime=822.14 cores=1,3,5,7
/proc/22030/task/22042/stat : 22042 (plingeling) R 22028 22030 21414 0 -1 4202560 442697 1259 0 0 82157 57 0 0 25 0 5 0 654911461 1945206784 446361 33554432000 4194304 4417452 140736591993600 18446744073709551615 4354515 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22030/tid=22043] ppid=22028 vsize=1899616 CPUtime=822.14 cores=1,3,5,7
/proc/22030/task/22043/stat : 22043 (plingeling) R 22028 22030 21414 0 -1 4202560 558875 1259 0 0 82146 68 0 0 25 0 5 0 654911461 1945206784 446361 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22030/tid=22044] ppid=22028 vsize=1899616 CPUtime=812.71 cores=1,3,5,7
/proc/22030/task/22044/stat : 22044 (plingeling) R 22028 22030 21414 0 -1 4202560 544048 1259 0 0 81178 93 0 0 25 0 5 0 654911461 1945206784 446361 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364958 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3279.33
Current children cumulated vsize (KiB) 1899616

[startup+882.301 s]
/proc/loadavg: 8.15 8.12 8.03 10/363 22068
/proc/meminfo: memFree=17561096/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=2061352 CPUtime=3518.62 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 2055934 1259 0 0 351572 290 0 0 24 0 5 0 654911444 2110824448 486763 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 515338 486763 180 55 0 513224 0
[pid=22030/tid=22041] ppid=22028 vsize=2061352 CPUtime=882.15 cores=1,3,5,7
/proc/22030/task/22041/stat : 22041 (plingeling) R 22028 22030 21414 0 -1 4202560 460636 1259 0 0 88151 64 0 0 25 0 5 0 654911461 2110824448 486763 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22030/tid=22042] ppid=22028 vsize=2061352 CPUtime=882.15 cores=1,3,5,7
/proc/22030/task/22042/stat : 22042 (plingeling) R 22028 22030 21414 0 -1 4202560 467273 1259 0 0 88155 60 0 0 25 0 5 0 654911461 2110824448 486763 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22030/tid=22043] ppid=22028 vsize=2061352 CPUtime=882.14 cores=1,3,5,7
/proc/22030/task/22043/stat : 22043 (plingeling) R 22028 22030 21414 0 -1 4202560 558875 1259 0 0 88146 68 0 0 25 0 5 0 654911461 2110824448 486763 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22030/tid=22044] ppid=22028 vsize=2061352 CPUtime=872 cores=1,3,5,7
/proc/22030/task/22044/stat : 22044 (plingeling) R 22028 22030 21414 0 -1 4202560 560432 1259 0 0 87104 96 0 0 25 0 5 0 654911461 2110824448 486763 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3518.62
Current children cumulated vsize (KiB) 2061352

[startup+942.301 s]
/proc/loadavg: 8.14 8.13 8.04 9/363 22069
/proc/meminfo: memFree=17076604/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=2319604 CPUtime=3757.94 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 2161457 1259 0 0 375486 308 0 0 24 0 5 0 654911444 2375274496 551326 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 579901 551326 180 55 0 577787 0
[pid=22030/tid=22041] ppid=22028 vsize=2319604 CPUtime=942.14 cores=1,3,5,7
/proc/22030/task/22041/stat : 22041 (plingeling) R 22028 22030 21414 0 -1 4202560 476283 1259 0 0 94149 65 0 0 25 0 5 0 654911461 2375274496 551326 33554432000 4194304 4417452 140736591993600 18446744073709551615 4305857 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22030/tid=22042] ppid=22028 vsize=2319604 CPUtime=942.15 cores=1,3,5,7
/proc/22030/task/22042/stat : 22042 (plingeling) R 22028 22030 21414 0 -1 4202560 483657 1259 0 0 94153 62 0 0 25 0 5 0 654911461 2375274496 551326 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22030/tid=22043] ppid=22028 vsize=2319604 CPUtime=942.15 cores=1,3,5,7
/proc/22030/task/22043/stat : 22043 (plingeling) R 22028 22030 21414 0 -1 4202560 586783 1259 0 0 94142 73 0 0 25 0 5 0 654911461 2375274496 551326 33554432000 4194304 4417452 140736591993600 18446744073709551615 4366657 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22030/tid=22044] ppid=22028 vsize=2319604 CPUtime=931.3 cores=1,3,5,7
/proc/22030/task/22044/stat : 22044 (plingeling) R 22028 22030 21414 0 -1 4202560 606016 1259 0 0 93025 105 0 0 25 0 5 0 654911461 2375274496 551326 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3757.94
Current children cumulated vsize (KiB) 2319604

[startup+1002.3 s]
/proc/loadavg: 8.10 8.11 8.04 10/363 22071
/proc/meminfo: memFree=16584916/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=2524016 CPUtime=3997.24 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 2323582 1259 0 0 399394 330 0 0 24 0 5 0 654911444 2584592384 599652 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 631004 599652 180 55 0 628890 0
[pid=22030/tid=22041] ppid=22028 vsize=2524016 CPUtime=1002.15 cores=1,3,5,7
/proc/22030/task/22041/stat : 22041 (plingeling) R 22028 22030 21414 0 -1 4202560 547873 1259 0 0 100139 76 0 0 25 0 5 0 654911461 2584592384 599652 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22030/tid=22042] ppid=22028 vsize=2524016 CPUtime=1002.15 cores=1,3,5,7
/proc/22030/task/22042/stat : 22042 (plingeling) R 22028 22030 21414 0 -1 4202560 526804 1259 0 0 100147 68 0 0 25 0 5 0 654911461 2584592384 599652 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22030/tid=22043] ppid=22028 vsize=2524016 CPUtime=1002.15 cores=1,3,5,7
/proc/22030/task/22043/stat : 22043 (plingeling) R 22028 22030 21414 0 -1 4202560 627743 1259 0 0 100137 78 0 0 25 0 5 0 654911461 2584592384 599652 33554432000 4194304 4417452 140736591993600 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22030/tid=22044] ppid=22028 vsize=2524016 CPUtime=990.6 cores=1,3,5,7
/proc/22030/task/22044/stat : 22044 (plingeling) R 22028 22030 21414 0 -1 4202560 612444 1259 0 0 98954 106 0 0 25 0 5 0 654911461 2584592384 599652 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3997.24
Current children cumulated vsize (KiB) 2524016

[startup+1062.3 s]
/proc/loadavg: 8.03 8.09 8.03 10/363 22073
/proc/meminfo: memFree=16522752/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=2538016 CPUtime=4236.55 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 2410597 1259 0 0 423311 344 0 0 24 0 5 0 654911444 2598928384 605928 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 634504 605928 180 55 0 632390 0
[pid=22030/tid=22041] ppid=22028 vsize=2538016 CPUtime=1062.15 cores=1,3,5,7
/proc/22030/task/22041/stat : 22041 (plingeling) R 22028 22030 21414 0 -1 4202560 572449 1259 0 0 106135 80 0 0 25 0 5 0 654911461 2598928384 605928 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22030/tid=22042] ppid=22028 vsize=2538016 CPUtime=1062.15 cores=1,3,5,7
/proc/22030/task/22042/stat : 22042 (plingeling) R 22028 22030 21414 0 -1 4202560 534996 1259 0 0 106146 69 0 0 25 0 5 0 654911461 2598928384 605928 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22030/tid=22043] ppid=22028 vsize=2538016 CPUtime=1062.15 cores=1,3,5,7
/proc/22030/task/22043/stat : 22043 (plingeling) R 22028 22030 21414 0 -1 4202560 665606 1259 0 0 106132 83 0 0 25 0 5 0 654911461 2598928384 605928 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22030/tid=22044] ppid=22028 vsize=2538016 CPUtime=1049.91 cores=1,3,5,7
/proc/22030/task/22044/stat : 22044 (plingeling) R 22028 22030 21414 0 -1 4202560 628828 1259 0 0 104881 110 0 0 25 0 5 0 654911461 2598928384 605928 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4236.55
Current children cumulated vsize (KiB) 2538016

[startup+1122.3 s]
/proc/loadavg: 8.05 8.08 8.03 10/363 22074
/proc/meminfo: memFree=16252888/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=2659680 CPUtime=4475.85 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 2509749 1259 0 0 447228 357 0 0 24 0 5 0 654911444 2723512320 636344 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 664920 636344 180 55 0 662806 0
[pid=22030/tid=22041] ppid=22028 vsize=2659680 CPUtime=1122.16 cores=1,3,5,7
/proc/22030/task/22041/stat : 22041 (plingeling) R 22028 22030 21414 0 -1 4202560 597025 1259 0 0 112133 83 0 0 25 0 5 0 654911461 2723512320 636344 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22030/tid=22042] ppid=22028 vsize=2659680 CPUtime=1122.16 cores=1,3,5,7
/proc/22030/task/22042/stat : 22042 (plingeling) R 22028 22030 21414 0 -1 4202560 543188 1259 0 0 112146 70 0 0 25 0 5 0 654911461 2723512320 636344 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364558 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22030/tid=22043] ppid=22028 vsize=2659680 CPUtime=1122.15 cores=1,3,5,7
/proc/22030/task/22043/stat : 22043 (plingeling) R 22028 22030 21414 0 -1 4202560 731990 1259 0 0 112123 92 0 0 25 0 5 0 654911461 2723512320 636344 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22030/tid=22044] ppid=22028 vsize=2659680 CPUtime=1109.2 cores=1,3,5,7
/proc/22030/task/22044/stat : 22044 (plingeling) R 22028 22030 21414 0 -1 4202560 628828 1259 0 0 110810 110 0 0 25 0 5 0 654911461 2723512320 636344 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4475.85
Current children cumulated vsize (KiB) 2659680

[startup+1182.3 s]
/proc/loadavg: 8.05 8.08 8.03 10/363 22076
/proc/meminfo: memFree=17091584/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=1701060 CPUtime=4715.16 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 2589672 1259 0 0 471145 371 0 0 24 0 5 0 654911444 1741885440 396690 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 425265 396690 180 55 0 423151 0
[pid=22030/tid=22041] ppid=22028 vsize=1701060 CPUtime=1182.16 cores=1,3,5,7
/proc/22030/task/22041/stat : 22041 (plingeling) R 22028 22030 21414 0 -1 4202560 621721 1259 0 0 118129 87 0 0 25 0 5 0 654911461 1741885440 396690 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22030/tid=22042] ppid=22028 vsize=1701060 CPUtime=1182.16 cores=1,3,5,7
/proc/22030/task/22042/stat : 22042 (plingeling) R 22028 22030 21414 0 -1 4202560 543188 1259 0 0 118146 70 0 0 25 0 5 0 654911461 1741885440 396690 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22030/tid=22043] ppid=22028 vsize=1701060 CPUtime=1182.16 cores=1,3,5,7
/proc/22030/task/22043/stat : 22043 (plingeling) R 22028 22030 21414 0 -1 4202560 782814 1259 0 0 118117 99 0 0 25 0 5 0 654911461 1741885440 396690 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22030/tid=22044] ppid=22028 vsize=1701060 CPUtime=1168.5 cores=1,3,5,7
/proc/22030/task/22044/stat : 22044 (plingeling) R 22028 22030 21414 0 -1 4202560 633231 1259 0 0 116737 113 0 0 25 0 5 0 654911461 1741885440 396690 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4715.16
Current children cumulated vsize (KiB) 1701060



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.6 s]
/proc/loadavg: 7.85 8.04 8.02 5/358 22077
/proc/meminfo: memFree=19623324/32951124 swapFree=65581052/67111528
[pid=22030] ppid=22028 vsize=1741024 CPUtime=4800.11 cores=1,3,5,7
/proc/22030/stat : 22030 (plingeling) S 22028 22030 21414 0 -1 4202496 2624239 1259 0 0 479637 374 0 0 24 0 5 0 654911444 1782808576 406681 33554432000 4194304 4417452 140736591993600 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/22030/statm: 435256 406681 180 55 0 433142 0
[pid=22030/tid=22041] ppid=22028 vsize=1741024 CPUtime=1203.47 cores=1,3,5,7
/proc/22030/task/22041/stat : 22041 (plingeling) R 22028 22030 21414 0 -1 4202560 630211 1259 0 0 120259 88 0 0 25 0 5 0 654911461 1782808576 406681 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22030/tid=22042] ppid=22028 vsize=1741024 CPUtime=1203.47 cores=1,3,5,7
/proc/22030/task/22042/stat : 22042 (plingeling) R 22028 22030 21414 0 -1 4202560 543188 1259 0 0 120277 70 0 0 25 0 5 0 654911461 1782808576 406681 33554432000 4194304 4417452 140736591993600 18446744073709551615 4307843 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22030/tid=22043] ppid=22028 vsize=1741024 CPUtime=1203.46 cores=1,3,5,7
/proc/22030/task/22043/stat : 22043 (plingeling) R 22028 22030 21414 0 -1 4202560 808891 1259 0 0 120245 101 0 0 25 0 5 0 654911461 1782808576 406681 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22030/tid=22044] ppid=22028 vsize=1741024 CPUtime=1189.54 cores=1,3,5,7
/proc/22030/task/22044/stat : 22044 (plingeling) R 22028 22030 21414 0 -1 4202560 633231 1259 0 0 118841 113 0 0 25 0 5 0 654911461 1782808576 406681 33554432000 4194304 4417452 140736591993600 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.11
Current children cumulated vsize (KiB) 1741024

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.67
CPU time (s): 4800.22
CPU user time (s): 4796.41
CPU system time (s): 3.80742
CPU usage (%): 398.797
Max. virtual memory (cumulated for all children) (KiB): 2692448

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4796.41
system time used= 3.80742
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2625498
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 107
involuntary context switches= 36408

runsolver used 3.99039 second user time and 9.95649 second system time

The end

Launcher Data 

Begin job on node137 at 2011-05-15 06:18:57
IDJOB=3370900
IDBENCH=88223
IDSOLVER=1848
FILE ID=node137/3370900-1305433137
RUNJOBID= node137-1305433137-22006
PBS_JOBID= 13324606
Free space on /tmp= 73792 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m42-q663-n40-p15-s72490337.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3370900-1305433137/watcher-3370900-1305433137 -o /tmp/evaluation-result-3370900-1305433137/solver-3370900-1305433137 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3370900-1305433137.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= ab8e40d31c79e7a71114e3126a46874f
RANDOM SEED=1250668192

node137.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21253328 kB
Buffers:        292544 kB
Cached:        3736260 kB
SwapCached:      36328 kB
Active:        1667652 kB
Inactive:      9804680 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21253328 kB
SwapTotal:    67111528 kB
SwapFree:     65581052 kB
Dirty:            5936 kB
Writeback:           0 kB
AnonPages:     7417020 kB
Mapped:          19500 kB
Slab:           121208 kB
PageTables:      38776 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 75919220 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73796 MiB
End job on node137 at 2011-05-15 06:39:03