Trace number 3386019

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 3894.84 1300.09

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m32-q326.25-n30-p15-s44266159.cnf
MD5SUM0906ad3a7678a4284d484fb4b8c72f61
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.414936
Satisfiable
(Un)Satisfiability was proved
Number of variables792
Number of clauses67279
Sum of the clauses size135290
Maximum clause length31
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 267249
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 530

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386019-1305583750/watcher-3386019-1305583750 -o /tmp/evaluation-result-3386019-1305583750/solver-3386019-1305583750 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3386019-1305583750.cnf -threads=4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 9.54 9.98 9.52 4/386 4463
/proc/meminfo: memFree=20927936/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=20440 CPUtime=0 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 617 0 0 0 0 0 0 0 25 0 1 0 669972961 20930560 461 33554432000 4194304 4625164 140733860731712 18446744073709551615 228122713856 0 0 4096 2 0 0 0 17 3 0 0 0
/proc/4463/statm: 5110 461 296 106 0 613 0

[startup+0.0173171 s]
/proc/loadavg: 9.54 9.98 9.52 4/386 4463
/proc/meminfo: memFree=20927936/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=23312 CPUtime=0.01 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 1428 0 0 0 1 0 0 0 25 0 1 0 669972961 23871488 896 33554432000 4194304 4625164 140733860731712 18446744073709551615 228122411477 0 0 4096 2 0 0 0 17 3 0 0 0
/proc/4463/statm: 5828 912 298 106 0 1331 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 23312

[startup+0.100574 s]
/proc/loadavg: 9.54 9.98 9.52 4/386 4463
/proc/meminfo: memFree=20927936/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=254932 CPUtime=0.25 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 3182 0 0 0 25 0 0 0 25 0 4 0 669972961 261050368 2618 33554432000 4194304 4625164 140733860731712 18446744073709551615 228122411216 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/4463/statm: 63733 2618 333 106 0 59236 0
Current children cumulated CPU time (s) 0.25
Current children cumulated vsize (KiB) 254932

[startup+0.377215 s]
/proc/loadavg: 9.54 9.98 9.52 4/386 4463
/proc/meminfo: memFree=20927936/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=262932 CPUtime=1.08 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 8588 0 0 0 107 1 0 0 25 0 4 0 669972961 269242368 5756 33554432000 4194304 4625164 140733860731712 18446744073709551615 228122757303 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/4463/statm: 65733 5756 333 106 0 61236 0
Current children cumulated CPU time (s) 1.08
Current children cumulated vsize (KiB) 262932

[startup+0.774787 s]
/proc/loadavg: 9.54 9.98 9.52 4/386 4463
/proc/meminfo: memFree=20927936/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=271828 CPUtime=2.26 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 18670 0 0 0 224 2 0 0 25 0 4 0 669972961 278351872 11901 33554432000 4194304 4625164 140733860731712 18446744073709551615 4360561 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/4463/statm: 67957 11901 361 106 0 63460 0
Current children cumulated CPU time (s) 2.26
Current children cumulated vsize (KiB) 271828

[startup+1.50101 s]
/proc/loadavg: 9.54 9.98 9.52 4/386 4463
/proc/meminfo: memFree=20927936/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=277588 CPUtime=4.44 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 24483 0 0 0 441 3 0 0 18 0 4 0 669972961 284250112 14611 33554432000 4194304 4625164 140733860731712 18446744073709551615 4393110 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/4463/statm: 69397 14611 399 106 0 64900 0
Current children cumulated CPU time (s) 4.44
Current children cumulated vsize (KiB) 277588

[startup+3.10072 s]
/proc/loadavg: 9.54 9.98 9.52 11/396 4476
/proc/meminfo: memFree=20790128/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=281148 CPUtime=9.21 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 28084 0 0 0 918 3 0 0 19 0 4 0 669972961 287895552 18192 33554432000 4194304 4625164 140733860731712 18446744073709551615 4512054 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/4463/statm: 70287 18192 471 106 0 65790 0
[pid=4463/tid=4465] ppid=4461 vsize=281148 CPUtime=3.01 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 4144 0 0 0 301 0 0 0 23 0 4 0 669972963 287895552 18192 33554432000 4194304 4625164 140733860731712 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=281148 CPUtime=2.96 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 3652 0 0 0 296 0 0 0 25 0 4 0 669972963 287895552 18192 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=281148 CPUtime=2 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 2010 0 0 0 200 0 0 0 19 0 4 0 669972963 287895552 18192 33554432000 4194304 4625164 140733860731712 18446744073709551615 4547675 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 9.21
Current children cumulated vsize (KiB) 281148

[startup+6.30115 s]
/proc/loadavg: 9.57 9.98 9.52 11/396 4476
/proc/meminfo: memFree=20734340/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=281412 CPUtime=18.77 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 35767 0 0 0 1873 4 0 0 22 0 4 0 669972961 288165888 23549 33554432000 4194304 4625164 140733860731712 18446744073709551615 4513492 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/4463/statm: 70353 23549 494 106 0 65856 0
[pid=4463/tid=4465] ppid=4461 vsize=281412 CPUtime=6.16 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 6935 0 0 0 616 0 0 0 25 0 4 0 669972963 288165888 23549 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=281412 CPUtime=6.16 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 7736 0 0 0 615 1 0 0 25 0 4 0 669972963 288165888 23549 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=281412 CPUtime=3.6 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 2812 0 0 0 360 0 0 0 22 0 4 0 669972963 288165888 23549 33554432000 4194304 4625164 140733860731712 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 18.77
Current children cumulated vsize (KiB) 281412

[startup+12.715 s]
/proc/loadavg: 9.61 9.98 9.52 11/396 4476
/proc/meminfo: memFree=20664432/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=309612 CPUtime=37.92 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 51350 0 0 0 3785 7 0 0 25 0 4 0 669972961 317042688 30662 33554432000 4194304 4625164 140733860731712 18446744073709551615 4512444 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/4463/statm: 77403 30662 729 106 0 72906 0
[pid=4463/tid=4465] ppid=4461 vsize=309612 CPUtime=12.5 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 10845 0 0 0 1249 1 0 0 25 0 4 0 669972963 317042688 30662 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351901 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=309612 CPUtime=12.57 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 11987 0 0 0 1255 2 0 0 25 0 4 0 669972963 317042688 30662 33554432000 4194304 4625164 140733860731712 18446744073709551615 4352002 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=309612 CPUtime=6.8 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 10230 0 0 0 679 1 0 0 25 0 4 0 669972963 317042688 30662 33554432000 4194304 4625164 140733860731712 18446744073709551615 4352005 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37.92
Current children cumulated vsize (KiB) 309612

[startup+25.5007 s]
/proc/loadavg: 9.77 10.00 9.54 11/396 4477
/proc/meminfo: memFree=20572800/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=333160 CPUtime=76.13 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 89905 0 0 0 7598 15 0 0 25 0 4 0 669972961 341155840 39998 33554432000 4194304 4625164 140733860731712 18446744073709551615 4519319 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/4463/statm: 83290 39998 497 106 0 78793 0
[pid=4463/tid=4465] ppid=4461 vsize=333160 CPUtime=23.03 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 17398 0 0 0 2300 3 0 0 25 0 4 0 669972963 341155840 39998 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=333160 CPUtime=25.35 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 18953 0 0 0 2531 4 0 0 25 0 4 0 669972963 341155840 39998 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=333160 CPUtime=13.13 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 35042 0 0 0 1309 4 0 0 25 0 4 0 669972963 341155840 39998 33554432000 4194304 4625164 140733860731712 18446744073709551615 4579376 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 76.13
Current children cumulated vsize (KiB) 333160

[startup+51.1011 s]
/proc/loadavg: 9.91 10.01 9.55 11/396 4477
/proc/meminfo: memFree=20376172/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=366080 CPUtime=152.63 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 174312 0 0 0 15235 28 0 0 25 0 4 0 669972961 374865920 51872 33554432000 4194304 4625164 140733860731712 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/4463/statm: 91520 51872 346 106 0 87023 0
[pid=4463/tid=4465] ppid=4461 vsize=366080 CPUtime=38.78 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 25775 0 0 0 3874 4 0 0 25 0 4 0 669972963 374865920 51878 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=366080 CPUtime=50.96 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 29488 0 0 0 5090 6 0 0 25 0 4 0 669972963 374865920 51878 33554432000 4194304 4625164 140733860731712 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=366080 CPUtime=26.04 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 98803 0 0 0 2591 13 0 0 25 0 4 0 669972963 374865920 51878 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 152.63
Current children cumulated vsize (KiB) 366080

[startup+102.301 s]
/proc/loadavg: 10.09 10.04 9.58 11/396 4479
/proc/meminfo: memFree=19989248/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=515268 CPUtime=305.54 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 530006 0 0 0 30471 83 0 0 25 0 4 0 669972961 527634432 81247 33554432000 4194304 4625164 140733860731712 18446744073709551615 4393112 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/4463/statm: 128817 81247 399 106 0 124320 0
[pid=4463/tid=4465] ppid=4461 vsize=515268 CPUtime=65.88 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 39030 0 0 0 6581 7 0 0 25 0 4 0 669972963 527634432 81247 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=515268 CPUtime=90.86 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 40885 0 0 0 9077 9 0 0 25 0 4 0 669972963 527634432 81247 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351811 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=515268 CPUtime=61.99 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 396023 0 0 0 6141 58 0 0 25 0 4 0 669972963 527634432 81247 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351987 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 305.54
Current children cumulated vsize (KiB) 515268

[startup+162.315 s]
/proc/loadavg: 10.09 10.04 9.61 11/396 4480
/proc/meminfo: memFree=19640004/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=668156 CPUtime=484.8 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 864335 0 0 0 48346 134 0 0 25 0 4 0 669972961 684191744 109897 33554432000 4194304 4625164 140733860731712 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/4463/statm: 167039 109897 372 106 0 162542 0
[pid=4463/tid=4465] ppid=4461 vsize=668156 CPUtime=96.12 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 44823 0 0 0 9602 10 0 0 25 0 4 0 669972963 684191744 109897 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=668156 CPUtime=150.11 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 61837 0 0 0 14998 13 0 0 25 0 4 0 669972963 684191744 109897 33554432000 4194304 4625164 140733860731712 18446744073709551615 4352014 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=668156 CPUtime=98.19 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 679683 0 0 0 9719 100 0 0 25 0 4 0 669972963 684191744 109897 33554432000 4194304 4625164 140733860731712 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 484.8
Current children cumulated vsize (KiB) 668156

[startup+222.301 s]
/proc/loadavg: 10.14 10.06 9.64 11/396 4482
/proc/meminfo: memFree=19339372/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=729100 CPUtime=664.01 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 896992 0 0 0 66258 143 0 0 25 0 4 0 669972961 746598400 129763 33554432000 4194304 4625164 140733860731712 18446744073709551615 4512789 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/4463/statm: 182275 129763 376 106 0 177778 0
[pid=4463/tid=4465] ppid=4461 vsize=729100 CPUtime=125.89 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 60564 0 0 0 12576 13 0 0 25 0 4 0 669972963 746598400 129763 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=729100 CPUtime=194.38 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 66234 0 0 0 19423 15 0 0 25 0 4 0 669972963 746598400 129763 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351777 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=729100 CPUtime=151.55 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 687837 0 0 0 15053 102 0 0 25 0 4 0 669972963 746598400 129763 33554432000 4194304 4625164 140733860731712 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 664.01
Current children cumulated vsize (KiB) 729100

[startup+282.3 s]
/proc/loadavg: 10.21 10.09 9.68 11/396 4484
/proc/meminfo: memFree=19090308/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=886396 CPUtime=843.23 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 1010783 0 0 0 84158 165 0 0 25 0 4 0 669972961 907669504 158989 33554432000 4194304 4625164 140733860731712 18446744073709551615 4393110 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/4463/statm: 221599 158989 444 106 0 217102 0
[pid=4463/tid=4465] ppid=4461 vsize=886396 CPUtime=168.31 cores=1,3,5,7

################
# More data... #
################

[pid=4463] ppid=4461 vsize=1424556 CPUtime=2097.89 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 2756495 0 0 0 209334 455 0 0 25 0 4 0 669972961 1458745344 272289 33554432000 4194304 4625164 140733860731712 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/4463/statm: 356139 272289 350 106 0 351642 0
[pid=4463/tid=4465] ppid=4461 vsize=1424556 CPUtime=421.7 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 105370 0 0 0 42134 36 0 0 25 0 4 0 669972963 1458745344 272289 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=1424556 CPUtime=601 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 136578 0 0 0 60064 36 0 0 25 0 4 0 669972963 1458745344 272289 33554432000 4194304 4625164 140733860731712 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=1424556 CPUtime=485.83 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 2075058 0 0 0 48275 308 0 0 25 0 4 0 669972963 1458745344 272289 33554432000 4194304 4625164 140733860731712 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2097.89
Current children cumulated vsize (KiB) 1424556

[startup+762.301 s]
/proc/loadavg: 10.13 10.15 9.86 11/396 4496
/proc/meminfo: memFree=18107112/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=1633152 CPUtime=2282.17 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 3217821 0 0 0 227695 522 0 0 25 0 4 0 669972961 1672347648 309409 33554432000 4194304 4625164 140733860731712 18446744073709551615 4393051 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/4463/statm: 408288 309409 352 106 0 403791 0
[pid=4463/tid=4465] ppid=4461 vsize=1633152 CPUtime=481.68 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 138437 0 0 0 48127 41 0 0 25 0 4 0 669972963 1672347648 309409 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=1633152 CPUtime=634.21 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 141791 0 0 0 63383 38 0 0 25 0 4 0 669972963 1672347648 309409 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=1633152 CPUtime=517.42 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 2380299 0 0 0 51391 351 0 0 25 0 4 0 669972963 1672347648 309409 33554432000 4194304 4625164 140733860731712 18446744073709551615 4546832 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2282.17
Current children cumulated vsize (KiB) 1633152

[startup+822.315 s]
/proc/loadavg: 10.23 10.16 9.89 11/396 4498
/proc/meminfo: memFree=18031652/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=1743560 CPUtime=2461.44 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 3562944 0 0 0 245564 580 0 0 25 0 4 0 669972961 1785405440 320923 33554432000 4194304 4625164 140733860731712 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/4463/statm: 435890 320923 346 106 0 431393 0
[pid=4463/tid=4465] ppid=4461 vsize=1743560 CPUtime=541.69 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 142433 0 0 0 54125 44 0 0 25 0 4 0 669972963 1785405440 320923 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=1743560 CPUtime=664.67 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 143529 0 0 0 66427 40 0 0 25 0 4 0 669972963 1785405440 320923 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=1743560 CPUtime=549.91 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 2685832 0 0 0 54592 399 0 0 25 0 4 0 669972963 1785405440 320923 33554432000 4194304 4625164 140733860731712 18446744073709551615 4552354 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2461.44
Current children cumulated vsize (KiB) 1743560

[startup+882.301 s]
/proc/loadavg: 10.08 10.13 9.89 11/396 4500
/proc/meminfo: memFree=17971068/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=1749896 CPUtime=2640.64 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 3878110 0 0 0 263437 627 0 0 25 0 4 0 669972961 1791893504 330968 33554432000 4194304 4625164 140733860731712 18446744073709551615 4511936 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/4463/statm: 437474 330968 349 106 0 432977 0
[pid=4463/tid=4465] ppid=4461 vsize=1749896 CPUtime=601.68 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 146131 0 0 0 60123 45 0 0 25 0 4 0 669972963 1791893504 330968 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351987 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=1749896 CPUtime=694.31 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 145975 0 0 0 69389 42 0 0 25 0 4 0 669972963 1791893504 330968 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=1749896 CPUtime=580.49 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 2948785 0 0 0 57614 435 0 0 25 0 4 0 669972963 1791893504 330968 33554432000 4194304 4625164 140733860731712 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2640.64
Current children cumulated vsize (KiB) 1749896

[startup+942.3 s]
/proc/loadavg: 10.09 10.12 9.90 11/396 4501
/proc/meminfo: memFree=17897452/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=1760984 CPUtime=2819.87 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 4189710 0 0 0 281307 680 0 0 25 0 4 0 669972961 1803247616 339037 33554432000 4194304 4625164 140733860731712 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/4463/statm: 440246 339037 346 106 0 435749 0
[pid=4463/tid=4465] ppid=4461 vsize=1760984 CPUtime=660.94 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 150137 0 0 0 66046 48 0 0 25 0 4 0 669972963 1803247616 339037 33554432000 4194304 4625164 140733860731712 18446744073709551615 4340318 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=1760984 CPUtime=724.76 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 147334 0 0 0 72433 43 0 0 25 0 4 0 669972963 1803247616 339037 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=1760984 CPUtime=610 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 3195805 0 0 0 60529 471 0 0 25 0 4 0 669972963 1803247616 339037 33554432000 4194304 4625164 140733860731712 18446744073709551615 228122381274 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2819.87
Current children cumulated vsize (KiB) 1760984

[startup+1002.3 s]
/proc/loadavg: 10.09 10.11 9.91 11/396 4503
/proc/meminfo: memFree=17836520/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=1771544 CPUtime=3004.04 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 4545914 0 0 0 299672 732 0 0 25 0 4 0 669972961 1814061056 349546 33554432000 4194304 4625164 140733860731712 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/4463/statm: 442886 349546 354 106 0 438389 0
[pid=4463/tid=4465] ppid=4461 vsize=1771544 CPUtime=718.7 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 153550 0 0 0 71821 49 0 0 25 0 4 0 669972963 1814061056 349546 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=1771544 CPUtime=761.38 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 149682 0 0 0 76093 45 0 0 25 0 4 0 669972963 1814061056 349546 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=1771544 CPUtime=644.6 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 3488425 0 0 0 63948 512 0 0 25 0 4 0 669972963 1814061056 349546 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3004.04
Current children cumulated vsize (KiB) 1771544

[startup+1062.3 s]
/proc/loadavg: 10.08 10.10 9.92 11/396 4504
/proc/meminfo: memFree=17777544/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=1781180 CPUtime=3183.27 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 4896258 0 0 0 317538 789 0 0 25 0 4 0 669972961 1823928320 356742 33554432000 4194304 4625164 140733860731712 18446744073709551615 4519388 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/4463/statm: 445295 356742 346 106 0 440798 0
[pid=4463/tid=4465] ppid=4461 vsize=1781180 CPUtime=777.25 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 157442 0 0 0 77673 52 0 0 25 0 4 0 669972963 1823928320 356742 33554432000 4194304 4625164 140733860731712 18446744073709551615 4352072 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=1781180 CPUtime=792.99 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 150913 0 0 0 79252 47 0 0 25 0 4 0 669972963 1823928320 356742 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=1781180 CPUtime=675.88 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 3774977 0 0 0 67032 556 0 0 25 0 4 0 669972963 1823928320 356742 33554432000 4194304 4625164 140733860731712 18446744073709551615 228122757479 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3183.27
Current children cumulated vsize (KiB) 1781180

[startup+1122.3 s]
/proc/loadavg: 10.13 10.11 9.93 11/396 4506
/proc/meminfo: memFree=17682348/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=1873152 CPUtime=3362.51 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 5277227 0 0 0 335404 847 0 0 25 0 4 0 669972961 1918107648 368852 33554432000 4194304 4625164 140733860731712 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/4463/statm: 468288 368852 351 106 0 463791 0
[pid=4463/tid=4465] ppid=4461 vsize=1873152 CPUtime=837.23 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 160450 0 0 0 83669 54 0 0 25 0 4 0 669972963 1918107648 368852 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=1873152 CPUtime=823.18 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 197864 0 0 0 82266 52 0 0 25 0 4 0 669972963 1918107648 368852 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=1873152 CPUtime=705.71 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 4040960 0 0 0 69976 595 0 0 25 0 4 0 669972963 1918107648 368852 33554432000 4194304 4625164 140733860731712 18446744073709551615 4579598 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3362.51
Current children cumulated vsize (KiB) 1873152

[startup+1182.3 s]
/proc/loadavg: 10.14 10.12 9.94 11/396 4508
/proc/meminfo: memFree=17597472/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=1963568 CPUtime=3541.75 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 5671085 0 0 0 353265 910 0 0 25 0 4 0 669972961 2010693632 385918 33554432000 4194304 4625164 140733860731712 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/4463/statm: 490892 385918 351 106 0 486395 0
[pid=4463/tid=4465] ppid=4461 vsize=1963568 CPUtime=897.23 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 209013 0 0 0 89662 61 0 0 25 0 4 0 669972963 2010693632 385918 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351901 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=1963568 CPUtime=853.22 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 200518 0 0 0 85267 55 0 0 25 0 4 0 669972963 2010693632 385918 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=1963568 CPUtime=741.19 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 4317753 0 0 0 73481 638 0 0 25 0 4 0 669972963 2010693632 385918 33554432000 4194304 4625164 140733860731712 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3541.75
Current children cumulated vsize (KiB) 1963568

[startup+1242.3 s]
/proc/loadavg: 10.10 10.11 9.94 11/396 4509
/proc/meminfo: memFree=17524384/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=1972808 CPUtime=3722.36 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 5974063 0 0 0 371273 963 0 0 25 0 4 0 669972961 2020155392 394742 33554432000 4194304 4625164 140733860731712 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/4463/statm: 493202 394742 349 106 0 488705 0
[pid=4463/tid=4465] ppid=4461 vsize=1972808 CPUtime=956.57 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 212009 0 0 0 95594 63 0 0 25 0 4 0 669972963 2020155392 394742 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=1972808 CPUtime=883.87 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 202536 0 0 0 88330 57 0 0 25 0 4 0 669972963 2020155392 394742 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=1972808 CPUtime=772.61 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 4548874 0 0 0 76586 675 0 0 25 0 4 0 669972963 2020155392 394742 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3722.36
Current children cumulated vsize (KiB) 1972808



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 10.07 10.10 9.95 11/396 4511
/proc/meminfo: memFree=17468404/32951124 swapFree=49826708/67111528
[pid=4463] ppid=4461 vsize=1980992 CPUtime=3894.74 cores=1,3,5,7
/proc/4463/stat : 4463 (SArTagnan) R 4461 4463 3578 0 -1 4202496 6284035 0 0 0 388460 1014 0 0 25 0 4 0 669972961 2028535808 403787 33554432000 4194304 4625164 140733860731712 18446744073709551615 4511967 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/4463/statm: 495248 403787 346 106 0 490751 0
[pid=4463/tid=4465] ppid=4461 vsize=1980992 CPUtime=1014.29 cores=1,3,5,7
/proc/4463/task/4465/stat : 4465 (SArTagnan) R 4461 4463 3578 0 -1 4202560 215311 0 0 0 101364 65 0 0 25 0 4 0 669972963 2028535808 403787 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=4463/tid=4466] ppid=4461 vsize=1980992 CPUtime=912.72 cores=1,3,5,7
/proc/4463/task/4466/stat : 4466 (SArTagnan) R 4461 4463 3578 0 -1 4202560 204455 0 0 0 91213 59 0 0 25 0 4 0 669972963 2028535808 403787 33554432000 4194304 4625164 140733860731712 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=4463/tid=4467] ppid=4461 vsize=1980992 CPUtime=801.47 cores=1,3,5,7
/proc/4463/task/4467/stat : 4467 (SArTagnan) R 4461 4463 3578 0 -1 4202560 4792356 0 0 0 79436 711 0 0 25 0 4 0 669972963 2028535808 403787 33554432000 4194304 4625164 140733860731712 18446744073709551615 4547216 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3894.74
Current children cumulated vsize (KiB) 1980992

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1300.09
CPU time (s): 3894.84
CPU user time (s): 3884.64
CPU system time (s): 10.2064
CPU usage (%): 299.583
Max. virtual memory (cumulated for all children) (KiB): 2122948

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3884.64
system time used= 10.2064
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6284035
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3237
involuntary context switches= 32327

runsolver used 4.22436 second user time and 12.3361 second system time

The end

Launcher Data 

Begin job on node138 at 2011-05-17 00:09:10
IDJOB=3386019
IDBENCH=88224
IDSOLVER=1871
FILE ID=node138/3386019-1305583750
RUNJOBID= node138-1305582447-4341
PBS_JOBID= 13350210
Free space on /tmp= 73440 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m32-q326.25-n30-p15-s44266159.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386019-1305583750/watcher-3386019-1305583750 -o /tmp/evaluation-result-3386019-1305583750/solver-3386019-1305583750 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3386019-1305583750.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 0906ad3a7678a4284d484fb4b8c72f61
RANDOM SEED=1375868095

node138.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      20928092 kB
Buffers:          6824 kB
Cached:          96220 kB
SwapCached:      70912 kB
Active:        1812032 kB
Inactive:      9985768 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      20928092 kB
SwapTotal:    67111528 kB
SwapFree:     49826708 kB
Dirty:            2760 kB
Writeback:          20 kB
AnonPages:    11633524 kB
Mapped:          11332 kB
Slab:            73036 kB
PageTables:      86260 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 96029084 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73436 MiB
End job on node138 at 2011-05-17 00:30:52