Trace number 3258250

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.39 1201.75

General information on the benchmark

NameCRAFTED/rbsat/random/unforced/
rbsat-v1375c111739g4.cnf
MD5SUMb2f1edd9e4045a375bcfedb2d5e5bd0e
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark51.2682
Satisfiable
(Un)Satisfiability was proved
Number of variables1375
Number of clauses111739
Sum of the clauses size224743
Maximum clause length25
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2111684
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 555

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3258250-1304049350/watcher-3258250-1304049350 -o /tmp/evaluation-result-3258250-1304049350/solver-3258250-1304049350 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3258250-1304049350.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.82 5.51 6.35 2/177 3771
/proc/meminfo: memFree=31786836/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=9132 CPUtime=0 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) R 3769 3771 3183 0 -1 4202496 363 1257 0 0 0 0 0 0 22 0 1 0 497350634 9351168 292 33554432000 4194304 4417332 140735878027136 18446744073709551615 270338383616 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/3771/statm: 2283 292 146 55 0 169 0

[startup+0.00995499 s]
/proc/loadavg: 3.82 5.51 6.35 2/177 3771
/proc/meminfo: memFree=31786836/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=9528 CPUtime=0 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) R 3769 3771 3183 0 -1 4202496 472 1257 0 0 0 0 0 0 22 0 1 0 497350634 9756672 401 33554432000 4194304 4417332 140735878027136 18446744073709551615 4231277 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/3771/statm: 2382 401 146 55 0 268 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9528

[startup+0.177873 s]
/proc/loadavg: 3.82 5.51 6.35 2/177 3771
/proc/meminfo: memFree=31786836/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=72832 CPUtime=0.47 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 6903 1257 0 0 47 0 0 0 22 0 5 0 497350634 74579968 6024 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 18208 6024 177 55 0 16094 0
Current children cumulated CPU time (s) 0.47
Current children cumulated vsize (KiB) 72832

[startup+0.30085 s]
/proc/loadavg: 3.82 5.51 6.35 2/177 3771
/proc/meminfo: memFree=31786836/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=135232 CPUtime=0.96 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 9446 1257 0 0 95 1 0 0 22 0 5 0 497350634 138477568 5259 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 33808 5259 179 55 0 31694 0
Current children cumulated CPU time (s) 0.96
Current children cumulated vsize (KiB) 135232

[startup+0.700768 s]
/proc/loadavg: 3.82 5.51 6.35 2/177 3771
/proc/meminfo: memFree=31786836/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=148224 CPUtime=2.56 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 13537 1257 0 0 255 1 0 0 22 0 5 0 497350634 151781376 8480 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 37056 8480 181 55 0 34942 0
Current children cumulated CPU time (s) 2.56
Current children cumulated vsize (KiB) 148224

[startup+1.50061 s]
/proc/loadavg: 4.24 5.57 6.37 10/186 3785
/proc/meminfo: memFree=31721796/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=154412 CPUtime=5.75 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 17393 1257 0 0 574 1 0 0 22 0 5 0 497350634 158117888 10027 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 38603 10027 181 55 0 36489 0
[pid=3771/tid=3782] ppid=3769 vsize=154412 CPUtime=1.42 cores=1,3,5,7
/proc/3771/task/3782/stat : 3782 (plingeling) R 3769 3771 3183 0 -1 4202560 1669 1257 0 0 142 0 0 0 25 0 5 0 497350641 158117888 10027 33554432000 4194304 4417332 140735878027136 18446744073709551615 4249193 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3771/tid=3783] ppid=3769 vsize=154412 CPUtime=1.41 cores=1,3,5,7
/proc/3771/task/3783/stat : 3783 (plingeling) R 3769 3771 3183 0 -1 4202560 1800 1257 0 0 141 0 0 0 25 0 5 0 497350641 158117888 10027 33554432000 4194304 4417332 140735878027136 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3771/tid=3784] ppid=3769 vsize=154412 CPUtime=1.42 cores=1,3,5,7
/proc/3771/task/3784/stat : 3784 (plingeling) R 3769 3771 3183 0 -1 4202560 1311 1257 0 0 142 0 0 0 25 0 5 0 497350641 158117888 10027 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3771/tid=3785] ppid=3769 vsize=154412 CPUtime=1.41 cores=1,3,5,7
/proc/3771/task/3785/stat : 3785 (plingeling) R 3769 3771 3183 0 -1 4202560 8035 1257 0 0 141 0 0 0 25 0 5 0 497350641 158117888 10027 33554432000 4194304 4417332 140735878027136 18446744073709551615 4358020 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.75
Current children cumulated vsize (KiB) 154412

[startup+3.10029 s]
/proc/loadavg: 4.24 5.57 6.37 10/186 3785
/proc/meminfo: memFree=31710388/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=159076 CPUtime=12.15 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 26167 1257 0 0 1212 3 0 0 22 0 5 0 497350634 162893824 11426 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 39769 11426 181 55 0 37655 0
[pid=3771/tid=3782] ppid=3769 vsize=159076 CPUtime=3.02 cores=1,3,5,7
/proc/3771/task/3782/stat : 3782 (plingeling) R 3769 3771 3183 0 -1 4202560 1669 1257 0 0 302 0 0 0 25 0 5 0 497350641 162893824 11426 33554432000 4194304 4417332 140735878027136 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3771/tid=3783] ppid=3769 vsize=159076 CPUtime=3.01 cores=1,3,5,7
/proc/3771/task/3783/stat : 3783 (plingeling) R 3769 3771 3183 0 -1 4202560 2516 1257 0 0 301 0 0 0 25 0 5 0 497350641 162893824 11426 33554432000 4194304 4417332 140735878027136 18446744073709551615 4249216 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3771/tid=3784] ppid=3769 vsize=159076 CPUtime=3.02 cores=1,3,5,7
/proc/3771/task/3784/stat : 3784 (plingeling) R 3769 3771 3183 0 -1 4202560 1311 1257 0 0 302 0 0 0 25 0 5 0 497350641 162893824 11426 33554432000 4194304 4417332 140735878027136 18446744073709551615 4250762 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3771/tid=3785] ppid=3769 vsize=159076 CPUtime=3 cores=1,3,5,7
/proc/3771/task/3785/stat : 3785 (plingeling) R 3769 3771 3183 0 -1 4202560 16093 1257 0 0 299 1 0 0 25 0 5 0 497350641 162893824 11426 33554432000 4194304 4417332 140735878027136 18446744073709551615 4354428 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.15
Current children cumulated vsize (KiB) 159076

[startup+6.30066 s]
/proc/loadavg: 4.24 5.57 6.37 9/186 3786
/proc/meminfo: memFree=31695052/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=180780 CPUtime=24.94 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 37094 1257 0 0 2490 4 0 0 22 0 5 0 497350634 185118720 16632 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 45195 16632 181 55 0 43081 0
[pid=3771/tid=3782] ppid=3769 vsize=180780 CPUtime=6.22 cores=1,3,5,7
/proc/3771/task/3782/stat : 3782 (plingeling) R 3769 3771 3183 0 -1 4202560 1669 1257 0 0 622 0 0 0 25 0 5 0 497350641 185118720 16632 33554432000 4194304 4417332 140735878027136 18446744073709551615 4336643 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3771/tid=3783] ppid=3769 vsize=180780 CPUtime=6.21 cores=1,3,5,7
/proc/3771/task/3783/stat : 3783 (plingeling) R 3769 3771 3183 0 -1 4202560 2516 1257 0 0 621 0 0 0 25 0 5 0 497350641 185118720 16632 33554432000 4194304 4417332 140735878027136 18446744073709551615 4249171 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3771/tid=3784] ppid=3769 vsize=180780 CPUtime=6.22 cores=1,3,5,7
/proc/3771/task/3784/stat : 3784 (plingeling) R 3769 3771 3183 0 -1 4202560 1629 1257 0 0 622 0 0 0 25 0 5 0 497350641 185118720 16632 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3771/tid=3785] ppid=3769 vsize=180780 CPUtime=6.19 cores=1,3,5,7
/proc/3771/task/3785/stat : 3785 (plingeling) R 3769 3771 3183 0 -1 4202560 26702 1257 0 0 617 2 0 0 25 0 5 0 497350641 185118720 16632 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.94
Current children cumulated vsize (KiB) 180780

[startup+12.7004 s]
/proc/loadavg: 4.81 5.65 6.39 10/186 3786
/proc/meminfo: memFree=31625404/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=190564 CPUtime=50.51 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 58498 1257 0 0 5044 7 0 0 22 0 5 0 497350634 195137536 19078 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 47641 19078 181 55 0 45527 0
[pid=3771/tid=3782] ppid=3769 vsize=190564 CPUtime=12.62 cores=1,3,5,7
/proc/3771/task/3782/stat : 3782 (plingeling) R 3769 3771 3183 0 -1 4202560 3100 1257 0 0 1262 0 0 0 25 0 5 0 497350641 195137536 19078 33554432000 4194304 4417332 140735878027136 18446744073709551615 4249110 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3771/tid=3783] ppid=3769 vsize=190564 CPUtime=12.61 cores=1,3,5,7
/proc/3771/task/3783/stat : 3783 (plingeling) R 3769 3771 3183 0 -1 4202560 3231 1257 0 0 1261 0 0 0 25 0 5 0 497350641 195137536 19078 33554432000 4194304 4417332 140735878027136 18446744073709551615 4258263 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3771/tid=3784] ppid=3769 vsize=190564 CPUtime=12.62 cores=1,3,5,7
/proc/3771/task/3784/stat : 3784 (plingeling) R 3769 3771 3183 0 -1 4202560 2533 1257 0 0 1262 0 0 0 25 0 5 0 497350641 195137536 19078 33554432000 4194304 4417332 140735878027136 18446744073709551615 4249924 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3771/tid=3785] ppid=3769 vsize=190564 CPUtime=12.56 cores=1,3,5,7
/proc/3771/task/3785/stat : 3785 (plingeling) R 3769 3771 3183 0 -1 4202560 45056 1257 0 0 1251 5 0 0 25 0 5 0 497350641 195137536 19078 33554432000 4194304 4417332 140735878027136 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.51
Current children cumulated vsize (KiB) 190564

[startup+25.5009 s]
/proc/loadavg: 5.30 5.73 6.40 10/186 3786
/proc/meminfo: memFree=31571932/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=233100 CPUtime=101.64 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 89823 1257 0 0 10153 11 0 0 22 0 5 0 497350634 238694400 29712 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 58275 29712 181 55 0 56161 0
[pid=3771/tid=3782] ppid=3769 vsize=233100 CPUtime=25.42 cores=1,3,5,7
/proc/3771/task/3782/stat : 3782 (plingeling) R 3769 3771 3183 0 -1 4202560 3649 1257 0 0 2542 0 0 0 25 0 5 0 497350641 238694400 29712 33554432000 4194304 4417332 140735878027136 18446744073709551615 4254964 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3771/tid=3783] ppid=3769 vsize=233100 CPUtime=25.41 cores=1,3,5,7
/proc/3771/task/3783/stat : 3783 (plingeling) R 3769 3771 3183 0 -1 4202560 3553 1257 0 0 2541 0 0 0 25 0 5 0 497350641 238694400 29712 33554432000 4194304 4417332 140735878027136 18446744073709551615 4250306 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3771/tid=3784] ppid=3769 vsize=233100 CPUtime=25.42 cores=1,3,5,7
/proc/3771/task/3784/stat : 3784 (plingeling) R 3769 3771 3183 0 -1 4202560 4380 1257 0 0 2542 0 0 0 25 0 5 0 497350641 238694400 29712 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3771/tid=3785] ppid=3769 vsize=233100 CPUtime=25.3 cores=1,3,5,7
/proc/3771/task/3785/stat : 3785 (plingeling) R 3769 3771 3183 0 -1 4202560 73663 1257 0 0 2521 9 0 0 25 0 5 0 497350641 238694400 29712 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364722 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.64
Current children cumulated vsize (KiB) 233100

[startup+51.1008 s]
/proc/loadavg: 6.22 5.91 6.44 9/186 3787
/proc/meminfo: memFree=31422720/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=300044 CPUtime=203.92 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 135015 1257 0 0 20375 17 0 0 22 0 5 0 497350634 307245056 46448 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 75011 46448 181 55 0 72897 0
[pid=3771/tid=3782] ppid=3769 vsize=300044 CPUtime=51.01 cores=1,3,5,7
/proc/3771/task/3782/stat : 3782 (plingeling) R 3769 3771 3183 0 -1 4202560 5363 1257 0 0 5101 0 0 0 25 0 5 0 497350641 307245056 46448 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364753 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3771/tid=3783] ppid=3769 vsize=300044 CPUtime=51.01 cores=1,3,5,7
/proc/3771/task/3783/stat : 3783 (plingeling) R 3769 3771 3183 0 -1 4202560 4221 1257 0 0 5101 0 0 0 25 0 5 0 497350641 307245056 46448 33554432000 4194304 4417332 140735878027136 18446744073709551615 4309907 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3771/tid=3784] ppid=3769 vsize=300044 CPUtime=51.02 cores=1,3,5,7
/proc/3771/task/3784/stat : 3784 (plingeling) R 3769 3771 3183 0 -1 4202560 5517 1257 0 0 5102 0 0 0 25 0 5 0 497350641 307245056 46448 33554432000 4194304 4417332 140735878027136 18446744073709551615 4254855 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3771/tid=3785] ppid=3769 vsize=300044 CPUtime=50.77 cores=1,3,5,7
/proc/3771/task/3785/stat : 3785 (plingeling) R 3769 3771 3183 0 -1 4202560 115336 1257 0 0 5063 14 0 0 25 0 5 0 497350641 307245056 46448 33554432000 4194304 4417332 140735878027136 18446744073709551615 4366146 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.92
Current children cumulated vsize (KiB) 300044

[startup+102.307 s]
/proc/loadavg: 7.42 6.30 6.55 10/186 3788
/proc/meminfo: memFree=31282456/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=359728 CPUtime=408.48 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 203949 1257 0 0 40821 27 0 0 22 0 5 0 497350634 368361472 61369 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 89932 61369 181 55 0 87818 0
[pid=3771/tid=3782] ppid=3769 vsize=359728 CPUtime=102.22 cores=1,3,5,7
/proc/3771/task/3782/stat : 3782 (plingeling) R 3769 3771 3183 0 -1 4202560 7426 1257 0 0 10222 0 0 0 25 0 5 0 497350641 368361472 61369 33554432000 4194304 4417332 140735878027136 18446744073709551615 4310750 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3771/tid=3783] ppid=3769 vsize=359728 CPUtime=102.22 cores=1,3,5,7
/proc/3771/task/3783/stat : 3783 (plingeling) R 3769 3771 3183 0 -1 4202560 6269 1257 0 0 10221 1 0 0 25 0 5 0 497350641 368361472 61369 33554432000 4194304 4417332 140735878027136 18446744073709551615 4366440 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3771/tid=3784] ppid=3769 vsize=359728 CPUtime=102.22 cores=1,3,5,7
/proc/3771/task/3784/stat : 3784 (plingeling) R 3769 3771 3183 0 -1 4202560 6844 1257 0 0 10222 0 0 0 25 0 5 0 497350641 368361472 61369 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3771/tid=3785] ppid=3769 vsize=359728 CPUtime=101.7 cores=1,3,5,7
/proc/3771/task/3785/stat : 3785 (plingeling) R 3769 3771 3183 0 -1 4202560 178832 1257 0 0 10147 23 0 0 25 0 5 0 497350641 368361472 61369 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 408.48
Current children cumulated vsize (KiB) 359728

[startup+162.301 s]
/proc/loadavg: 7.82 6.62 6.64 10/186 3790
/proc/meminfo: memFree=30886500/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=567768 CPUtime=648.15 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 269368 1257 0 0 64778 37 0 0 22 0 5 0 497350634 581394432 113475 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 141942 113475 181 55 0 139828 0
[pid=3771/tid=3782] ppid=3769 vsize=567768 CPUtime=162.22 cores=1,3,5,7
/proc/3771/task/3782/stat : 3782 (plingeling) R 3769 3771 3183 0 -1 4202560 11998 1257 0 0 16221 1 0 0 25 0 5 0 497350641 581394432 113475 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3771/tid=3783] ppid=3769 vsize=567768 CPUtime=162.22 cores=1,3,5,7
/proc/3771/task/3783/stat : 3783 (plingeling) R 3769 3771 3183 0 -1 4202560 12543 1257 0 0 16220 2 0 0 25 0 5 0 497350641 581394432 113475 33554432000 4194304 4417332 140735878027136 18446744073709551615 4357160 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3771/tid=3784] ppid=3769 vsize=567768 CPUtime=162.22 cores=1,3,5,7
/proc/3771/task/3784/stat : 3784 (plingeling) R 3769 3771 3183 0 -1 4202560 14279 1257 0 0 16221 1 0 0 25 0 5 0 497350641 581394432 113475 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3771/tid=3785] ppid=3769 vsize=567768 CPUtime=161.39 cores=1,3,5,7
/proc/3771/task/3785/stat : 3785 (plingeling) R 3769 3771 3183 0 -1 4202560 225970 1257 0 0 16107 32 0 0 25 0 5 0 497350641 581394432 113475 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 648.15
Current children cumulated vsize (KiB) 567768


################
# More data... #
################

[pid=3771/tid=3783] ppid=3769 vsize=833788 CPUtime=762.25 cores=1,3,5,7
/proc/3771/task/3783/stat : 3783 (plingeling) R 3769 3771 3183 0 -1 4202560 32529 1257 0 0 76221 4 0 0 25 0 5 0 497350641 853798912 179916 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364973 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3771/tid=3784] ppid=3769 vsize=833788 CPUtime=762.25 cores=1,3,5,7
/proc/3771/task/3784/stat : 3784 (plingeling) R 3769 3771 3183 0 -1 4202560 55605 1257 0 0 76219 6 0 0 25 0 5 0 497350641 853798912 179916 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3771/tid=3785] ppid=3769 vsize=833788 CPUtime=758.19 cores=1,3,5,7
/proc/3771/task/3785/stat : 3785 (plingeling) R 3769 3771 3183 0 -1 4202560 579144 1257 0 0 75737 82 0 0 25 0 5 0 497350641 853798912 179916 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3045.03
Current children cumulated vsize (KiB) 833788

[startup+822.301 s]
/proc/loadavg: 8.13 7.95 7.37 10/186 3816
/proc/meminfo: memFree=30130244/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=917460 CPUtime=3284.69 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 734555 1257 0 0 328370 99 0 0 22 0 5 0 497350634 939479040 200802 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 229365 200802 181 55 0 227251 0
[pid=3771/tid=3782] ppid=3769 vsize=917460 CPUtime=822.24 cores=1,3,5,7
/proc/3771/task/3782/stat : 3782 (plingeling) R 3769 3771 3183 0 -1 4202560 30994 1257 0 0 82221 3 0 0 25 0 5 0 497350641 939479040 200802 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3771/tid=3783] ppid=3769 vsize=917460 CPUtime=822.25 cores=1,3,5,7
/proc/3771/task/3783/stat : 3783 (plingeling) R 3769 3771 3183 0 -1 4202560 37855 1257 0 0 82221 4 0 0 25 0 5 0 497350641 939479040 200802 33554432000 4194304 4417332 140735878027136 18446744073709551615 4365963 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3771/tid=3784] ppid=3769 vsize=917460 CPUtime=822.25 cores=1,3,5,7
/proc/3771/task/3784/stat : 3784 (plingeling) R 3769 3771 3183 0 -1 4202560 64372 1257 0 0 82218 7 0 0 25 0 5 0 497350641 939479040 200802 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364667 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3771/tid=3785] ppid=3769 vsize=917460 CPUtime=817.85 cores=1,3,5,7
/proc/3771/task/3785/stat : 3785 (plingeling) R 3769 3771 3183 0 -1 4202560 596756 1257 0 0 81702 83 0 0 25 0 5 0 497350641 939479040 200802 33554432000 4194304 4417332 140735878027136 18446744073709551615 4357111 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3284.69
Current children cumulated vsize (KiB) 917460

[startup+882.301 s]
/proc/loadavg: 8.13 7.99 7.42 10/186 3818
/proc/meminfo: memFree=30173360/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=906672 CPUtime=3524.37 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 743846 1257 0 0 352337 100 0 0 22 0 5 0 497350634 928432128 198105 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 226668 198105 181 55 0 224554 0
[pid=3771/tid=3782] ppid=3769 vsize=906672 CPUtime=882.24 cores=1,3,5,7
/proc/3771/task/3782/stat : 3782 (plingeling) R 3769 3771 3183 0 -1 4202560 31276 1257 0 0 88221 3 0 0 25 0 5 0 497350641 928432128 198105 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3771/tid=3783] ppid=3769 vsize=906672 CPUtime=882.25 cores=1,3,5,7
/proc/3771/task/3783/stat : 3783 (plingeling) R 3769 3771 3183 0 -1 4202560 37855 1257 0 0 88221 4 0 0 25 0 5 0 497350641 928432128 198105 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3771/tid=3784] ppid=3769 vsize=906672 CPUtime=882.25 cores=1,3,5,7
/proc/3771/task/3784/stat : 3784 (plingeling) R 3769 3771 3183 0 -1 4202560 73381 1257 0 0 88217 8 0 0 25 0 5 0 497350641 928432128 198105 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3771/tid=3785] ppid=3769 vsize=906672 CPUtime=877.52 cores=1,3,5,7
/proc/3771/task/3785/stat : 3785 (plingeling) R 3769 3771 3183 0 -1 4202560 596756 1257 0 0 87669 83 0 0 25 0 5 0 497350641 928432128 198105 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364973 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3524.37
Current children cumulated vsize (KiB) 906672

[startup+942.301 s]
/proc/loadavg: 8.13 8.01 7.46 10/186 3819
/proc/meminfo: memFree=29837668/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=1029616 CPUtime=3764.06 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 783550 1257 0 0 376299 107 0 0 22 0 5 0 497350634 1054326784 228841 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 257404 228841 181 55 0 255290 0
[pid=3771/tid=3782] ppid=3769 vsize=1029616 CPUtime=942.24 cores=1,3,5,7
/proc/3771/task/3782/stat : 3782 (plingeling) R 3769 3771 3183 0 -1 4202560 31276 1257 0 0 94221 3 0 0 25 0 5 0 497350641 1054326784 228841 33554432000 4194304 4417332 140735878027136 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3771/tid=3783] ppid=3769 vsize=1029616 CPUtime=942.25 cores=1,3,5,7
/proc/3771/task/3783/stat : 3783 (plingeling) R 3769 3771 3183 0 -1 4202560 43516 1257 0 0 94220 5 0 0 25 0 5 0 497350641 1054326784 228841 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3771/tid=3784] ppid=3769 vsize=1029616 CPUtime=942.25 cores=1,3,5,7
/proc/3771/task/3784/stat : 3784 (plingeling) R 3769 3771 3183 0 -1 4202560 79527 1257 0 0 94217 8 0 0 25 0 5 0 497350641 1054326784 228841 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3771/tid=3785] ppid=3769 vsize=1029616 CPUtime=937.21 cores=1,3,5,7
/proc/3771/task/3785/stat : 3785 (plingeling) R 3769 3771 3183 0 -1 4202560 624653 1257 0 0 93632 89 0 0 25 0 5 0 497350641 1054326784 228841 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3764.06
Current children cumulated vsize (KiB) 1029616

[startup+1002.3 s]
/proc/loadavg: 8.08 8.02 7.50 10/186 3821
/proc/meminfo: memFree=29780604/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=1077560 CPUtime=4003.76 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 817172 1257 0 0 400263 113 0 0 22 0 5 0 497350634 1103421440 240827 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 269390 240827 181 55 0 267276 0
[pid=3771/tid=3782] ppid=3769 vsize=1077560 CPUtime=1002.25 cores=1,3,5,7
/proc/3771/task/3782/stat : 3782 (plingeling) R 3769 3771 3183 0 -1 4202560 35372 1257 0 0 100221 4 0 0 25 0 5 0 497350641 1103421440 240827 33554432000 4194304 4417332 140735878027136 18446744073709551615 4357641 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3771/tid=3783] ppid=3769 vsize=1077560 CPUtime=1002.26 cores=1,3,5,7
/proc/3771/task/3783/stat : 3783 (plingeling) R 3769 3771 3183 0 -1 4202560 43516 1257 0 0 100221 5 0 0 25 0 5 0 497350641 1103421440 240827 33554432000 4194304 4417332 140735878027136 18446744073709551615 4344093 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3771/tid=3784] ppid=3769 vsize=1077560 CPUtime=1002.26 cores=1,3,5,7
/proc/3771/task/3784/stat : 3784 (plingeling) R 3769 3771 3183 0 -1 4202560 93558 1257 0 0 100216 10 0 0 25 0 5 0 497350641 1103421440 240827 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3771/tid=3785] ppid=3769 vsize=1077560 CPUtime=996.89 cores=1,3,5,7
/proc/3771/task/3785/stat : 3785 (plingeling) R 3769 3771 3183 0 -1 4202560 640148 1257 0 0 99597 92 0 0 25 0 5 0 497350641 1103421440 240827 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4003.76
Current children cumulated vsize (KiB) 1077560

[startup+1062.3 s]
/proc/loadavg: 8.13 8.05 7.54 10/186 3823
/proc/meminfo: memFree=29645160/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=1089700 CPUtime=4243.44 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 832468 1257 0 0 424229 115 0 0 22 0 5 0 497350634 1115852800 243862 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 272425 243862 181 55 0 270311 0
[pid=3771/tid=3782] ppid=3769 vsize=1089700 CPUtime=1062.25 cores=1,3,5,7
/proc/3771/task/3782/stat : 3782 (plingeling) R 3769 3771 3183 0 -1 4202560 37420 1257 0 0 106221 4 0 0 25 0 5 0 497350641 1115852800 243862 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3771/tid=3783] ppid=3769 vsize=1089700 CPUtime=1062.26 cores=1,3,5,7
/proc/3771/task/3783/stat : 3783 (plingeling) R 3769 3771 3183 0 -1 4202560 46722 1257 0 0 106221 5 0 0 25 0 5 0 497350641 1115852800 243862 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364750 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3771/tid=3784] ppid=3769 vsize=1089700 CPUtime=1062.26 cores=1,3,5,7
/proc/3771/task/3784/stat : 3784 (plingeling) R 3769 3771 3183 0 -1 4202560 102654 1257 0 0 106215 11 0 0 25 0 5 0 497350641 1115852800 243862 33554432000 4194304 4417332 140735878027136 18446744073709551615 4308010 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3771/tid=3785] ppid=3769 vsize=1089700 CPUtime=1056.56 cores=1,3,5,7
/proc/3771/task/3785/stat : 3785 (plingeling) R 3769 3771 3183 0 -1 4202560 641094 1257 0 0 105564 92 0 0 25 0 5 0 497350641 1115852800 243862 33554432000 4194304 4417332 140735878027136 18446744073709551615 4365525 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4243.44
Current children cumulated vsize (KiB) 1089700

[startup+1122.3 s]
/proc/loadavg: 8.08 8.05 7.57 10/186 3824
/proc/meminfo: memFree=29945456/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=1262488 CPUtime=4483.11 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 891615 1257 0 0 448186 125 0 0 22 0 5 0 497350634 1292787712 287059 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 315622 287059 181 55 0 313508 0
[pid=3771/tid=3782] ppid=3769 vsize=1262488 CPUtime=1122.25 cores=1,3,5,7
/proc/3771/task/3782/stat : 3782 (plingeling) R 3769 3771 3183 0 -1 4202560 37420 1257 0 0 112221 4 0 0 25 0 5 0 497350641 1292787712 287059 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3771/tid=3783] ppid=3769 vsize=1262488 CPUtime=1122.26 cores=1,3,5,7
/proc/3771/task/3783/stat : 3783 (plingeling) R 3769 3771 3183 0 -1 4202560 48770 1257 0 0 112221 5 0 0 25 0 5 0 497350641 1292787712 287059 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364753 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3771/tid=3784] ppid=3769 vsize=1262488 CPUtime=1122.26 cores=1,3,5,7
/proc/3771/task/3784/stat : 3784 (plingeling) R 3769 3771 3183 0 -1 4202560 109973 1257 0 0 112214 12 0 0 25 0 5 0 497350641 1292787712 287059 33554432000 4194304 4417332 140735878027136 18446744073709551615 4277540 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3771/tid=3785] ppid=3769 vsize=1262488 CPUtime=1116.24 cores=1,3,5,7
/proc/3771/task/3785/stat : 3785 (plingeling) R 3769 3771 3183 0 -1 4202560 690874 1257 0 0 111522 102 0 0 25 0 5 0 497350641 1292787712 287059 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4483.11
Current children cumulated vsize (KiB) 1262488

[startup+1182.3 s]
/proc/loadavg: 8.03 8.04 7.59 10/186 3826
/proc/meminfo: memFree=29865692/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=1247568 CPUtime=4722.8 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 904982 1257 0 0 472153 127 0 0 22 0 5 0 497350634 1277509632 283329 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 311892 283329 181 55 0 309778 0
[pid=3771/tid=3782] ppid=3769 vsize=1247568 CPUtime=1182.25 cores=1,3,5,7
/proc/3771/task/3782/stat : 3782 (plingeling) R 3769 3771 3183 0 -1 4202560 39468 1257 0 0 118221 4 0 0 25 0 5 0 497350641 1277509632 283329 33554432000 4194304 4417332 140735878027136 18446744073709551615 4343913 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3771/tid=3783] ppid=3769 vsize=1247568 CPUtime=1182.26 cores=1,3,5,7
/proc/3771/task/3783/stat : 3783 (plingeling) R 3769 3771 3183 0 -1 4202560 53669 1257 0 0 118220 6 0 0 25 0 5 0 497350641 1277509632 283329 33554432000 4194304 4417332 140735878027136 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3771/tid=3784] ppid=3769 vsize=1247568 CPUtime=1182.26 cores=1,3,5,7
/proc/3771/task/3784/stat : 3784 (plingeling) R 3769 3771 3183 0 -1 4202560 114502 1257 0 0 118214 12 0 0 25 0 5 0 497350641 1277509632 283329 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3771/tid=3785] ppid=3769 vsize=1247568 CPUtime=1175.92 cores=1,3,5,7
/proc/3771/task/3785/stat : 3785 (plingeling) R 3769 3771 3183 0 -1 4202560 692765 1257 0 0 117490 102 0 0 25 0 5 0 497350641 1277509632 283329 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4722.8
Current children cumulated vsize (KiB) 1247568



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 8.02 8.04 7.60 6/184 3827
/proc/meminfo: memFree=30337624/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=1280336 CPUtime=4800.32 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 915230 1257 0 0 479904 128 0 0 22 0 5 0 497350634 1311064064 291529 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 320084 291529 181 55 0 317970 0
[pid=3771/tid=3782] ppid=3769 vsize=1280336 CPUtime=1201.66 cores=1,3,5,7
/proc/3771/task/3782/stat : 3782 (plingeling) R 3769 3771 3183 0 -1 4202560 39468 1257 0 0 120162 4 0 0 25 0 5 0 497350641 1311064064 291529 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364899 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3771/tid=3783] ppid=3769 vsize=1280336 CPUtime=1201.67 cores=1,3,5,7
/proc/3771/task/3783/stat : 3783 (plingeling) R 3769 3771 3183 0 -1 4202560 55725 1257 0 0 120161 6 0 0 25 0 5 0 497350641 1311064064 291529 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3771/tid=3784] ppid=3769 vsize=1280336 CPUtime=1201.66 cores=1,3,5,7
/proc/3771/task/3784/stat : 3784 (plingeling) R 3769 3771 3183 0 -1 4202560 114502 1257 0 0 120154 12 0 0 25 0 5 0 497350641 1311064064 291529 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364990 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3771/tid=3785] ppid=3769 vsize=1280336 CPUtime=1195.21 cores=1,3,5,7
/proc/3771/task/3785/stat : 3785 (plingeling) R 3769 3771 3183 0 -1 4202560 700957 1257 0 0 119418 103 0 0 25 0 5 0 497350641 1311064064 291529 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.32
Current children cumulated vsize (KiB) 1280336

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 8.02 8.04 7.60 6/184 3827
/proc/meminfo: memFree=30337624/32951124 swapFree=67111356/67111528
[pid=3771] ppid=3769 vsize=1280336 CPUtime=4800.32 cores=1,3,5,7
/proc/3771/stat : 3771 (plingeling) S 3769 3771 3183 0 -1 4202496 915230 1257 0 0 479904 128 0 0 22 0 5 0 497350634 1311064064 291529 33554432000 4194304 4417332 140735878027136 18446744073709551615 270350186469 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/3771/statm: 320084 291529 181 55 0 317970 0
[pid=3771/tid=3782] ppid=3769 vsize=1280336 CPUtime=1201.66 cores=1,3,5,7
/proc/3771/task/3782/stat : 3782 (plingeling) R 3769 3771 3183 0 -1 4202560 39468 1257 0 0 120162 4 0 0 25 0 5 0 497350641 1311064064 291529 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364899 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3771/tid=3783] ppid=3769 vsize=1280336 CPUtime=1201.67 cores=1,3,5,7
/proc/3771/task/3783/stat : 3783 (plingeling) R 3769 3771 3183 0 -1 4202560 55725 1257 0 0 120161 6 0 0 25 0 5 0 497350641 1311064064 291529 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3771/tid=3784] ppid=3769 vsize=1280336 CPUtime=1201.66 cores=1,3,5,7
/proc/3771/task/3784/stat : 3784 (plingeling) R 3769 3771 3183 0 -1 4202560 114502 1257 0 0 120154 12 0 0 25 0 5 0 497350641 1311064064 291529 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364990 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3771/tid=3785] ppid=3769 vsize=1280336 CPUtime=1195.21 cores=1,3,5,7
/proc/3771/task/3785/stat : 3785 (plingeling) R 3769 3771 3183 0 -1 4202560 700957 1257 0 0 119418 103 0 0 25 0 5 0 497350641 1311064064 291529 33554432000 4194304 4417332 140735878027136 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.32
Current children cumulated vsize (KiB) 1280336

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.75
CPU time (s): 4800.39
CPU user time (s): 4799.06
CPU system time (s): 1.3298
CPU usage (%): 399.449
Max. virtual memory (cumulated for all children) (KiB): 1302948

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4799.06
system time used= 1.3298
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 916488
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 80
involuntary context switches= 35468

runsolver used 1.88371 second user time and 4.55031 second system time

The end

Launcher Data 

Begin job on node143 at 2011-04-29 05:55:50
IDJOB=3258250
IDBENCH=71317
IDSOLVER=1605
FILE ID=node143/3258250-1304049350
RUNJOBID= node143-1304049350-3747
PBS_JOBID= 13170399
Free space on /tmp= 72924 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT09/CRAFTED/rbsat/random/unforced/rbsat-v1375c111739g4.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3258250-1304049350/watcher-3258250-1304049350 -o /tmp/evaluation-result-3258250-1304049350/solver-3258250-1304049350 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3258250-1304049350.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= b2f1edd9e4045a375bcfedb2d5e5bd0e
RANDOM SEED=1924078049

node143.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      31786868 kB
Buffers:        181568 kB
Cached:         637304 kB
SwapCached:        168 kB
Active:         249716 kB
Inactive:       638492 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      31786868 kB
SwapTotal:    67111528 kB
SwapFree:     67111356 kB
Dirty:            2784 kB
Writeback:           0 kB
AnonPages:       69720 kB
Mapped:          15468 kB
Slab:           211840 kB
PageTables:       4760 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   209104 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72924 MiB
End job on node143 at 2011-04-29 06:15:54