Trace number 3322486

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 35694 5100.17

General information on the benchmark

Namecrafted/kullmann/VanDerWaerden/
VanderWaerden3k/VanDerWaerden_2-3-20_389.cnf
MD5SUMf105570f47bdca662c2d3d185f8dc1f8
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables389
Number of clauses41426
Sum of the clauses size188708
Maximum clause length20
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 337636
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 53790

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...386646/instance-3322486-1304386646.cnf
0.19/0.24	c Solving...
35693.82/5100.01	
35693.82/5100.01	*** INTERRUPTED! ***
35693.82/5100.01	*** Shutting down threads...
35693.96/5100.16	s UNKNOWN
35693.96/5100.16	
35693.96/5100.16	c INTERRUPTED   : 1
35693.96/5100.16	c Models        : 0+    
35693.96/5100.16	c Time          : 5100.014s (Solving: 5099.77s 1st Model: 0.00s Unsat: 0.00s)
35693.96/5100.16	c CPU Time      : 35693.860s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3322486-1304386646/watcher-3322486-1304386646 -o /tmp/evaluation-result-3322486-1304386646/solver-3322486-1304386646 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322486-1304386646.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.46 8.94 8.97 2/285 18694
/proc/meminfo: memFree=5071028/32951124 swapFree=67051472/67111528
[pid=18694] ppid=18692 vsize=22768 CPUtime=0 cores=0-7
/proc/18694/stat : 18694 (claspmt) R 18692 18694 18668 0 -1 4202496 955 0 0 0 0 0 0 0 22 0 1 0 550267355 23314432 848 33554432000 4194304 4964704 140733977311968 18446744073709551615 218370654416 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/18694/statm: 5692 861 386 189 0 566 0

[startup+0.0524349 s]
/proc/loadavg: 8.46 8.94 8.97 2/285 18694
/proc/meminfo: memFree=5071028/32951124 swapFree=67051472/67111528
[pid=18694] ppid=18692 vsize=25280 CPUtime=0.05 cores=0-7
/proc/18694/stat : 18694 (claspmt) R 18692 18694 18668 0 -1 4202496 1727 0 0 0 5 0 0 0 22 0 1 0 550267355 25886720 1482 33554432000 4194304 4964704 140733977311968 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/18694/statm: 6320 1482 399 189 0 1194 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 25280

[startup+0.100428 s]
/proc/loadavg: 8.46 8.94 8.97 2/285 18694
/proc/meminfo: memFree=5071028/32951124 swapFree=67051472/67111528
[pid=18694] ppid=18692 vsize=25280 CPUtime=0.09 cores=0-7
/proc/18694/stat : 18694 (claspmt) R 18692 18694 18668 0 -1 4202496 1727 0 0 0 9 0 0 0 22 0 1 0 550267355 25886720 1482 33554432000 4194304 4964704 140733977311968 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/18694/statm: 6320 1482 399 189 0 1194 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 25280

[startup+0.30039 s]
/proc/loadavg: 8.46 8.94 8.97 2/285 18694
/proc/meminfo: memFree=5071028/32951124 swapFree=67051472/67111528
[pid=18694] ppid=18692 vsize=253408 CPUtime=0.59 cores=0-7
/proc/18694/stat : 18694 (claspmt) R 18692 18694 18668 0 -1 4202496 2645 0 0 0 58 1 0 0 25 0 8 0 550267355 259489792 2313 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/18694/statm: 63352 2314 464 189 0 57701 0
Current children cumulated CPU time (s) 0.59
Current children cumulated vsize (KiB) 253408

[startup+0.700319 s]
/proc/loadavg: 8.46 8.94 8.97 2/285 18694
/proc/meminfo: memFree=5071028/32951124 swapFree=67051472/67111528
[pid=18694] ppid=18692 vsize=256356 CPUtime=3.39 cores=0-7
/proc/18694/stat : 18694 (claspmt) R 18692 18694 18668 0 -1 4202496 5708 0 0 0 337 2 0 0 25 0 8 0 550267355 262508544 5376 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/18694/statm: 64089 5377 464 189 0 58438 0
Current children cumulated CPU time (s) 3.39
Current children cumulated vsize (KiB) 256356

[startup+1.50118 s]
/proc/loadavg: 8.46 8.94 8.97 11/293 18702
/proc/meminfo: memFree=5047784/32951124 swapFree=67051472/67111528
[pid=18694] ppid=18692 vsize=259776 CPUtime=8.98 cores=0-7
/proc/18694/stat : 18694 (claspmt) R 18692 18694 18668 0 -1 4202496 8537 0 0 0 895 3 0 0 25 0 8 0 550267355 266010624 8205 33554432000 4194304 4964704 140733977311968 18446744073709551615 218371062303 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/18694/statm: 64944 8205 464 189 0 59293 0
[pid=18694/tid=18696] ppid=18692 vsize=259776 CPUtime=1.23 cores=0-7
/proc/18694/task/18696/stat : 18696 (claspmt) R 18692 18694 18668 0 -1 4202560 760 0 0 0 123 0 0 0 25 0 8 0 550267379 266010624 8205 33554432000 4194304 4964704 140733977311968 18446744073709551615 4627524 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18694/tid=18697] ppid=18692 vsize=259776 CPUtime=1.17 cores=0-7
/proc/18694/task/18697/stat : 18697 (claspmt) R 18692 18694 18668 0 -1 4202560 938 0 0 0 117 0 0 0 19 0 8 0 550267379 266010624 8205 33554432000 4194304 4964704 140733977311968 18446744073709551615 4668690 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18694/tid=18698] ppid=18692 vsize=259776 CPUtime=1.24 cores=0-7
/proc/18694/task/18698/stat : 18698 (claspmt) R 18692 18694 18668 0 -1 4202560 836 0 0 0 124 0 0 0 19 0 8 0 550267379 266010624 8205 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=18694/tid=18699] ppid=18692 vsize=259776 CPUtime=1.24 cores=0-7
/proc/18694/task/18699/stat : 18699 (claspmt) R 18692 18694 18668 0 -1 4202560 814 0 0 0 124 0 0 0 19 0 8 0 550267379 266010624 8205 33554432000 4194304 4964704 140733977311968 18446744073709551615 4668609 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18694/tid=18700] ppid=18692 vsize=259776 CPUtime=1.22 cores=0-7
/proc/18694/task/18700/stat : 18700 (claspmt) R 18692 18694 18668 0 -1 4202560 969 0 0 0 122 0 0 0 19 0 8 0 550267379 266010624 8205 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670255 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18694/tid=18701] ppid=18692 vsize=259776 CPUtime=1.24 cores=0-7
/proc/18694/task/18701/stat : 18701 (claspmt) R 18692 18694 18668 0 -1 4202560 974 0 0 0 124 0 0 0 18 0 8 0 550267379 266010624 8205 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670184 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18694/tid=18702] ppid=18692 vsize=259776 CPUtime=1.24 cores=0-7
/proc/18694/task/18702/stat : 18702 (claspmt) R 18692 18694 18668 0 -1 4202560 1265 0 0 0 124 0 0 0 19 0 8 0 550267379 266010624 8205 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 8.98
Current children cumulated vsize (KiB) 259776

[startup+3.10089 s]
/proc/loadavg: 8.51 8.94 8.97 10/293 18702
/proc/meminfo: memFree=5037444/32951124 swapFree=67051472/67111528
[pid=18694] ppid=18692 vsize=261112 CPUtime=20.18 cores=0-7
/proc/18694/stat : 18694 (claspmt) R 18692 18694 18668 0 -1 4202496 9952 0 0 0 2014 4 0 0 25 0 8 0 550267355 267378688 9620 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670255 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/18694/statm: 65278 9620 464 189 0 59627 0
[pid=18694/tid=18696] ppid=18692 vsize=261112 CPUtime=2.83 cores=0-7
/proc/18694/task/18696/stat : 18696 (claspmt) R 18692 18694 18668 0 -1 4202560 933 0 0 0 283 0 0 0 25 0 8 0 550267379 267378688 9620 33554432000 4194304 4964704 140733977311968 18446744073709551615 4627290 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18694/tid=18697] ppid=18692 vsize=261112 CPUtime=2.77 cores=0-7
/proc/18694/task/18697/stat : 18697 (claspmt) R 18692 18694 18668 0 -1 4202560 1171 0 0 0 277 0 0 0 23 0 8 0 550267379 267378688 9620 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670184 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18694/tid=18698] ppid=18692 vsize=261112 CPUtime=2.84 cores=0-7
/proc/18694/task/18698/stat : 18698 (claspmt) R 18692 18694 18668 0 -1 4202560 1024 0 0 0 284 0 0 0 23 0 8 0 550267379 267378688 9620 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=18694/tid=18699] ppid=18692 vsize=261112 CPUtime=2.84 cores=0-7
/proc/18694/task/18699/stat : 18699 (claspmt) R 18692 18694 18668 0 -1 4202560 952 0 0 0 284 0 0 0 23 0 8 0 550267379 267378688 9620 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670281 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18694/tid=18700] ppid=18692 vsize=261112 CPUtime=2.82 cores=0-7
/proc/18694/task/18700/stat : 18700 (claspmt) R 18692 18694 18668 0 -1 4202560 1133 0 0 0 282 0 0 0 23 0 8 0 550267379 267378688 9620 33554432000 4194304 4964704 140733977311968 18446744073709551615 4668609 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18694/tid=18701] ppid=18692 vsize=261112 CPUtime=2.29 cores=0-7
/proc/18694/task/18701/stat : 18701 (claspmt) R 18692 18694 18668 0 -1 4202560 1172 0 0 0 229 0 0 0 19 0 8 0 550267379 267378688 9620 33554432000 4194304 4964704 140733977311968 18446744073709551615 4653450 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18694/tid=18702] ppid=18692 vsize=261112 CPUtime=2.84 cores=0-7
/proc/18694/task/18702/stat : 18702 (claspmt) R 18692 18694 18668 0 -1 4202560 1554 0 0 0 284 0 0 0 23 0 8 0 550267379 267378688 9620 33554432000 4194304 4964704 140733977311968 18446744073709551615 4627928 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 20.18
Current children cumulated vsize (KiB) 261112

[startup+6.30033 s]
/proc/loadavg: 8.51 8.94 8.97 10/293 18702
/proc/meminfo: memFree=5028036/32951124 swapFree=67051472/67111528
[pid=18694] ppid=18692 vsize=264616 CPUtime=42.57 cores=0-7
/proc/18694/stat : 18694 (claspmt) R 18692 18694 18668 0 -1 4202496 12926 0 0 0 4251 6 0 0 25 0 8 0 550267355 270966784 12594 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/18694/statm: 66154 12594 466 189 0 60503 0
[pid=18694/tid=18696] ppid=18692 vsize=264616 CPUtime=5.28 cores=0-7
/proc/18694/task/18696/stat : 18696 (claspmt) R 18692 18694 18668 0 -1 4202560 1217 0 0 0 528 0 0 0 25 0 8 0 550267379 270966784 12594 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18694/tid=18697] ppid=18692 vsize=264616 CPUtime=5.97 cores=0-7
/proc/18694/task/18697/stat : 18697 (claspmt) R 18692 18694 18668 0 -1 4202560 1877 0 0 0 597 0 0 0 25 0 8 0 550267379 270966784 12594 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18694/tid=18698] ppid=18692 vsize=264616 CPUtime=5.67 cores=0-7
/proc/18694/task/18698/stat : 18698 (claspmt) R 18692 18694 18668 0 -1 4202560 1262 0 0 0 567 0 0 0 25 0 8 0 550267379 270966784 12594 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18694/tid=18699] ppid=18692 vsize=264616 CPUtime=6.04 cores=0-7
/proc/18694/task/18699/stat : 18699 (claspmt) R 18692 18694 18668 0 -1 4202560 1302 0 0 0 604 0 0 0 25 0 8 0 550267379 270966784 12594 33554432000 4194304 4964704 140733977311968 18446744073709551615 4627290 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18694/tid=18700] ppid=18692 vsize=264616 CPUtime=5.46 cores=0-7
/proc/18694/task/18700/stat : 18700 (claspmt) R 18692 18694 18668 0 -1 4202560 1485 0 0 0 546 0 0 0 25 0 8 0 550267379 270966784 12594 33554432000 4194304 4964704 140733977311968 18446744073709551615 4668668 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=18694/tid=18701] ppid=18692 vsize=264616 CPUtime=4.96 cores=0-7
/proc/18694/task/18701/stat : 18701 (claspmt) R 18692 18694 18668 0 -1 4202560 1543 0 0 0 496 0 0 0 25 0 8 0 550267379 270966784 12594 33554432000 4194304 4964704 140733977311968 18446744073709551615 218371062254 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=18694/tid=18702] ppid=18692 vsize=264616 CPUtime=5.24 cores=0-7
/proc/18694/task/18702/stat : 18702 (claspmt) R 18692 18694 18668 0 -1 4202560 1950 0 0 0 523 1 0 0 25 0 8 0 550267379 270966784 12594 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 42.57
Current children cumulated vsize (KiB) 264616

[startup+12.7012 s]
/proc/loadavg: 8.58 8.94 8.97 10/293 18702
/proc/meminfo: memFree=4994264/32951124 swapFree=67051472/67111528
[pid=18694] ppid=18692 vsize=466152 CPUtime=87.14 cores=0-7
/proc/18694/stat : 18694 (claspmt) R 18692 18694 18668 0 -1 4202496 21433 0 0 0 8703 11 0 0 18 0 8 0 550267355 477339648 21101 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/18694/statm: 116538 21101 466 189 0 110887 0
[pid=18694/tid=18696] ppid=18692 vsize=466152 CPUtime=11.44 cores=0-7
/proc/18694/task/18696/stat : 18696 (claspmt) R 18692 18694 18668 0 -1 4202560 1931 0 0 0 1144 0 0 0 25 0 8 0 550267379 477339648 21101 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670288 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18694/tid=18697] ppid=18692 vsize=466152 CPUtime=12.01 cores=0-7
/proc/18694/task/18697/stat : 18697 (claspmt) R 18692 18694 18668 0 -1 4202560 3552 0 0 0 1199 2 0 0 25 0 8 0 550267379 477339648 21101 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670357 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18694/tid=18698] ppid=18692 vsize=466152 CPUtime=11.57 cores=0-7
/proc/18694/task/18698/stat : 18698 (claspmt) R 18692 18694 18668 0 -1 4202560 2024 0 0 0 1157 0 0 0 25 0 8 0 550267379 477339648 21101 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18694/tid=18699] ppid=18692 vsize=466152 CPUtime=12.42 cores=0-7
/proc/18694/task/18699/stat : 18699 (claspmt) R 18692 18694 18668 0 -1 4202560 2232 0 0 0 1241 1 0 0 19 0 8 0 550267379 477339648 21101 33554432000 4194304 4964704 140733977311968 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18694/tid=18700] ppid=18692 vsize=466152 CPUtime=10.75 cores=0-7
/proc/18694/task/18700/stat : 18700 (claspmt) R 18692 18694 18668 0 -1 4202560 2330 0 0 0 1075 0 0 0 25 0 8 0 550267379 477339648 21101 33554432000 4194304 4964704 140733977311968 18446744073709551615 4668683 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18694/tid=18701] ppid=18692 vsize=466152 CPUtime=10.09 cores=0-7
/proc/18694/task/18701/stat : 18701 (claspmt) R 18692 18694 18668 0 -1 4202560 2698 0 0 0 1008 1 0 0 24 0 8 0 550267379 477339648 21101 33554432000 4194304 4964704 140733977311968 18446744073709551615 4321023 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=18694/tid=18702] ppid=18692 vsize=466152 CPUtime=11.08 cores=0-7
/proc/18694/task/18702/stat : 18702 (claspmt) R 18692 18694 18668 0 -1 4202560 3769 0 0 0 1106 2 0 0 25 0 8 0 550267379 477339648 21101 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670335 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 87.14
Current children cumulated vsize (KiB) 466152

[startup+25.5009 s]
/proc/loadavg: 8.72 8.96 8.97 10/293 18702
/proc/meminfo: memFree=4967020/32951124 swapFree=67051472/67111528
[pid=18694] ppid=18692 vsize=467100 CPUtime=176.74 cores=0-7
/proc/18694/stat : 18694 (claspmt) R 18692 18694 18668 0 -1 4202496 27856 0 0 0 17658 16 0 0 25 0 8 0 550267355 478310400 27343 33554432000 4194304 4964704 140733977311968 18446744073709551615 4342162 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/18694/statm: 116775 27343 466 189 0 111124 0
[pid=18694/tid=18696] ppid=18692 vsize=467100 CPUtime=24.25 cores=0-7
/proc/18694/task/18696/stat : 18696 (claspmt) R 18692 18694 18668 0 -1 4202560 2891 0 0 0 2424 1 0 0 25 0 8 0 550267379 478310400 27343 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18694/tid=18697] ppid=18692 vsize=467100 CPUtime=24.8 cores=0-7
/proc/18694/task/18697/stat : 18697 (claspmt) R 18692 18694 18668 0 -1 4202560 4668 0 0 0 2478 2 0 0 25 0 8 0 550267379 478310400 27343 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670574 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18694/tid=18698] ppid=18692 vsize=467100 CPUtime=22.87 cores=0-7
/proc/18694/task/18698/stat : 18698 (claspmt) R 18692 18694 18668 0 -1 4202560 2510 0 0 0 2286 1 0 0 25 0 8 0 550267379 478310400 27343 33554432000 4194304 4964704 140733977311968 18446744073709551615 4320976 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18694/tid=18699] ppid=18692 vsize=467100 CPUtime=25.22 cores=0-7
/proc/18694/task/18699/stat : 18699 (claspmt) R 18692 18694 18668 0 -1 4202560 2824 0 0 0 2520 2 0 0 25 0 8 0 550267379 478310400 27343 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18694/tid=18700] ppid=18692 vsize=467100 CPUtime=23.55 cores=0-7
/proc/18694/task/18700/stat : 18700 (claspmt) R 18692 18694 18668 0 -1 4202560 2997 0 0 0 2354 1 0 0 25 0 8 0 550267379 478310400 27343 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18694/tid=18701] ppid=18692 vsize=467100 CPUtime=16.88 cores=0-7
/proc/18694/task/18701/stat : 18701 (claspmt) R 18692 18694 18668 0 -1 4202560 3079 0 0 0 1687 1 0 0 25 0 8 0 550267379 478310400 27343 33554432000 4194304 4964704 140733977311968 18446744073709551615 4320848 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18694/tid=18702] ppid=18692 vsize=467100 CPUtime=21.38 cores=0-7
/proc/18694/task/18702/stat : 18702 (claspmt) R 18692 18694 18668 0 -1 4202560 5492 0 0 0 2135 3 0 0 25 0 8 0 550267379 478310400 27343 33554432000 4194304 4964704 140733977311968 18446744073709551615 4668673 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 176.74
Current children cumulated vsize (KiB) 467100

[startup+51.1004 s]
/proc/loadavg: 8.87 8.98 8.98 10/293 18703
/proc/meminfo: memFree=4937540/32951124 swapFree=67051472/67111528
[pid=18694] ppid=18692 vsize=470168 CPUtime=355.86 cores=0-7
/proc/18694/stat : 18694 (claspmt) R 18692 18694 18668 0 -1 4202496 35403 0 0 0 35563 23 0 0 25 0 8 0 550267355 481452032 34878 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/18694/statm: 117542 34878 466 189 0 111891 0
[pid=18694/tid=18696] ppid=18692 vsize=470168 CPUtime=41.01 cores=0-7
/proc/18694/task/18696/stat : 18696 (claspmt) R 18692 18694 18668 0 -1 4202560 4091 0 0 0 4099 2 0 0 25 0 8 0 550267379 481452032 34878 33554432000 4194304 4964704 140733977311968 18446744073709551615 4627365 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18694/tid=18697] ppid=18692 vsize=470168 CPUtime=42.1 cores=0-7
/proc/18694/task/18697/stat : 18697 (claspmt) R 18692 18694 18668 0 -1 4202560 5565 0 0 0 4207 3 0 0 25 0 8 0 550267379 481452032 34878 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670255 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18694/tid=18698] ppid=18692 vsize=470168 CPUtime=48.47 cores=0-7
/proc/18694/task/18698/stat : 18698 (claspmt) R 18692 18694 18668 0 -1 4202560 3679 0 0 0 4845 2 0 0 25 0 8 0 550267379 481452032 34878 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18694/tid=18699] ppid=18692 vsize=470168 CPUtime=49.88 cores=0-7
/proc/18694/task/18699/stat : 18699 (claspmt) R 18692 18694 18668 0 -1 4202560 3731 0 0 0 4985 3 0 0 25 0 8 0 550267379 481452032 34878 33554432000 4194304 4964704 140733977311968 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=18694/tid=18700] ppid=18692 vsize=470168 CPUtime=46.89 cores=0-7
/proc/18694/task/18700/stat : 18700 (claspmt) R 18692 18694 18668 0 -1 4202560 3847 0 0 0 4688 1 0 0 25 0 8 0 550267379 481452032 34878 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18694/tid=18701] ppid=18692 vsize=470168 CPUtime=40.33 cores=0-7
/proc/18694/task/18701/stat : 18701 (claspmt) R 18692 18694 18668 0 -1 4202560 4150 0 0 0 4031 2 0 0 25 0 8 0 550267379 481452032 34878 33554432000 4194304 4964704 140733977311968 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18694/tid=18702] ppid=18692 vsize=470168 CPUtime=46.96 cores=0-7
/proc/18694/task/18702/stat : 18702 (claspmt) R 18692 18694 18668 0 -1 4202560 6561 0 0 0 4692 4 0 0 25 0 8 0 550267379 481452032 34878 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670241 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 355.86

################
# More data... #
################

[pid=18694/tid=18697] ppid=18692 vsize=938952 CPUtime=4028.4 cores=0-7
/proc/18694/task/18697/stat : 18697 (claspmt) R 18692 18694 18668 0 -1 4202560 29172 0 0 0 402811 29 0 0 25 0 8 0 550267379 961486848 154296 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18694/tid=18698] ppid=18692 vsize=938952 CPUtime=4188.49 cores=0-7
/proc/18694/task/18698/stat : 18698 (claspmt) R 18692 18694 18668 0 -1 4202560 13933 0 0 0 418824 25 0 0 25 0 8 0 550267379 961486848 154296 33554432000 4194304 4964704 140733977311968 18446744073709551615 4321065 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18694/tid=18699] ppid=18692 vsize=938952 CPUtime=4355.37 cores=0-7
/proc/18694/task/18699/stat : 18699 (claspmt) R 18692 18694 18668 0 -1 4202560 17848 0 0 0 435512 25 0 0 25 0 8 0 550267379 961486848 154296 33554432000 4194304 4964704 140733977311968 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18694/tid=18700] ppid=18692 vsize=938952 CPUtime=4154.27 cores=0-7
/proc/18694/task/18700/stat : 18700 (claspmt) R 18692 18694 18668 0 -1 4202560 16839 0 0 0 415408 19 0 0 25 0 8 0 550267379 961486848 154296 33554432000 4194304 4964704 140733977311968 18446744073709551615 4321025 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18694/tid=18701] ppid=18692 vsize=938952 CPUtime=4249.35 cores=0-7
/proc/18694/task/18701/stat : 18701 (claspmt) R 18692 18694 18668 0 -1 4202560 15153 0 0 0 424909 26 0 0 25 0 8 0 550267379 961486848 154296 33554432000 4194304 4964704 140733977311968 18446744073709551615 4342228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18694/tid=18702] ppid=18692 vsize=938952 CPUtime=4139.84 cores=0-7
/proc/18694/task/18702/stat : 18702 (claspmt) R 18692 18694 18668 0 -1 4202560 29346 0 0 0 413937 47 0 0 25 0 8 0 550267379 961486848 154296 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 33471.2
Current children cumulated vsize (KiB) 938952

[startup+4842.3 s]
/proc/loadavg: 9.08 9.06 9.00 10/293 19341
/proc/meminfo: memFree=4448856/32951124 swapFree=67051472/67111528
[pid=18694] ppid=18692 vsize=938952 CPUtime=33890.6 cores=0-7
/proc/18694/stat : 18694 (claspmt) R 18692 18694 18668 0 -1 4202496 168405 0 0 0 3388818 243 0 0 25 0 8 0 550267355 961486848 155375 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/18694/statm: 234738 155375 466 189 0 229087 0
[pid=18694/tid=18696] ppid=18692 vsize=938952 CPUtime=4056.13 cores=0-7
/proc/18694/task/18696/stat : 18696 (claspmt) R 18692 18694 18668 0 -1 4202560 29028 0 0 0 405580 33 0 0 25 0 8 0 550267379 961486848 155375 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670243 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18694/tid=18697] ppid=18692 vsize=938952 CPUtime=4088.4 cores=0-7
/proc/18694/task/18697/stat : 18697 (claspmt) R 18692 18694 18668 0 -1 4202560 29253 0 0 0 408811 29 0 0 25 0 8 0 550267379 961486848 155375 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670215 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18694/tid=18698] ppid=18692 vsize=938952 CPUtime=4248.47 cores=0-7
/proc/18694/task/18698/stat : 18698 (claspmt) R 18692 18694 18668 0 -1 4202560 13933 0 0 0 424822 25 0 0 25 0 8 0 550267379 961486848 155375 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670366 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18694/tid=18699] ppid=18692 vsize=938952 CPUtime=4414.23 cores=0-7
/proc/18694/task/18699/stat : 18699 (claspmt) R 18692 18694 18668 0 -1 4202560 17909 0 0 0 441398 25 0 0 25 0 8 0 550267379 961486848 155375 33554432000 4194304 4964704 140733977311968 18446744073709551615 218371062254 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18694/tid=18700] ppid=18692 vsize=938952 CPUtime=4214.26 cores=0-7
/proc/18694/task/18700/stat : 18700 (claspmt) R 18692 18694 18668 0 -1 4202560 17215 0 0 0 421407 19 0 0 18 0 8 0 550267379 961486848 155375 33554432000 4194304 4964704 140733977311968 18446744073709551615 4627878 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18694/tid=18701] ppid=18692 vsize=938952 CPUtime=4281.28 cores=0-7
/proc/18694/task/18701/stat : 18701 (claspmt) R 18692 18694 18668 0 -1 4202560 15170 0 0 0 428102 26 0 0 25 0 8 0 550267379 961486848 155375 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18694/tid=18702] ppid=18692 vsize=938952 CPUtime=4199.85 cores=0-7
/proc/18694/task/18702/stat : 18702 (claspmt) R 18692 18694 18668 0 -1 4202560 29354 0 0 0 419937 48 0 0 25 0 8 0 550267379 961486848 155375 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 33890.6
Current children cumulated vsize (KiB) 938952

[startup+4902.3 s]
/proc/loadavg: 9.03 9.05 9.00 10/293 19342
/proc/meminfo: memFree=4447288/32951124 swapFree=67051472/67111528
[pid=18694] ppid=18692 vsize=938952 CPUtime=34311.7 cores=0-7
/proc/18694/stat : 18694 (claspmt) R 18692 18694 18668 0 -1 4202496 169027 0 0 0 3430930 244 0 0 25 0 8 0 550267355 961486848 155800 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/18694/statm: 234738 155800 466 189 0 229087 0
[pid=18694/tid=18696] ppid=18692 vsize=938952 CPUtime=4098.33 cores=0-7
/proc/18694/task/18696/stat : 18696 (claspmt) R 18692 18694 18668 0 -1 4202560 29359 0 0 0 409800 33 0 0 25 0 8 0 550267379 961486848 155800 33554432000 4194304 4964704 140733977311968 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18694/tid=18697] ppid=18692 vsize=938952 CPUtime=4148.4 cores=0-7
/proc/18694/task/18697/stat : 18697 (claspmt) R 18692 18694 18668 0 -1 4202560 29494 0 0 0 414811 29 0 0 25 0 8 0 550267379 961486848 155800 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18694/tid=18698] ppid=18692 vsize=938952 CPUtime=4308.46 cores=0-7
/proc/18694/task/18698/stat : 18698 (claspmt) R 18692 18694 18668 0 -1 4202560 13948 0 0 0 430820 26 0 0 25 0 8 0 550267379 961486848 155800 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670312 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18694/tid=18699] ppid=18692 vsize=938952 CPUtime=4450.47 cores=0-7
/proc/18694/task/18699/stat : 18699 (claspmt) R 18692 18694 18668 0 -1 4202560 17909 0 0 0 445022 25 0 0 25 0 8 0 550267379 961486848 155800 33554432000 4194304 4964704 140733977311968 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18694/tid=18700] ppid=18692 vsize=938952 CPUtime=4274.26 cores=0-7
/proc/18694/task/18700/stat : 18700 (claspmt) R 18692 18694 18668 0 -1 4202560 17215 0 0 0 427407 19 0 0 25 0 8 0 550267379 961486848 155800 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670243 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18694/tid=18701] ppid=18692 vsize=938952 CPUtime=4333.53 cores=0-7
/proc/18694/task/18701/stat : 18701 (claspmt) R 18692 18694 18668 0 -1 4202560 15180 0 0 0 433327 26 0 0 25 0 8 0 550267379 961486848 155800 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18694/tid=18702] ppid=18692 vsize=938952 CPUtime=4259.85 cores=0-7
/proc/18694/task/18702/stat : 18702 (claspmt) R 18692 18694 18668 0 -1 4202560 29379 0 0 0 425937 48 0 0 25 0 8 0 550267379 961486848 155800 33554432000 4194304 4964704 140733977311968 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 34311.7
Current children cumulated vsize (KiB) 938952

[startup+4962.3 s]
/proc/loadavg: 9.01 9.03 9.00 10/293 19344
/proc/meminfo: memFree=4443784/32951124 swapFree=67051472/67111528
[pid=18694] ppid=18692 vsize=939380 CPUtime=34731.2 cores=0-7
/proc/18694/stat : 18694 (claspmt) R 18692 18694 18668 0 -1 4202496 170091 0 0 0 3472871 247 0 0 25 0 8 0 550267355 961925120 156697 33554432000 4194304 4964704 140733977311968 18446744073709551615 4320909 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/18694/statm: 234845 156697 466 189 0 229194 0
[pid=18694/tid=18696] ppid=18692 vsize=939380 CPUtime=4133.43 cores=0-7
/proc/18694/task/18696/stat : 18696 (claspmt) R 18692 18694 18668 0 -1 4202560 29551 0 0 0 413310 33 0 0 25 0 8 0 550267379 961925120 156697 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18694/tid=18697] ppid=18692 vsize=939380 CPUtime=4208.4 cores=0-7
/proc/18694/task/18697/stat : 18697 (claspmt) R 18692 18694 18668 0 -1 4202560 29634 0 0 0 420810 30 0 0 25 0 8 0 550267379 961925120 156697 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670260 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18694/tid=18698] ppid=18692 vsize=939380 CPUtime=4368.45 cores=0-7
/proc/18694/task/18698/stat : 18698 (claspmt) R 18692 18694 18668 0 -1 4202560 14262 0 0 0 436818 27 0 0 25 0 8 0 550267379 961925120 156697 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670243 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18694/tid=18699] ppid=18692 vsize=939380 CPUtime=4508.47 cores=0-7
/proc/18694/task/18699/stat : 18699 (claspmt) R 18692 18694 18668 0 -1 4202560 18043 0 0 0 450822 25 0 0 25 0 8 0 550267379 961925120 156697 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18694/tid=18700] ppid=18692 vsize=939380 CPUtime=4334.26 cores=0-7
/proc/18694/task/18700/stat : 18700 (claspmt) R 18692 18694 18668 0 -1 4202560 17339 0 0 0 433407 19 0 0 25 0 8 0 550267379 961925120 156697 33554432000 4194304 4964704 140733977311968 18446744073709551615 4653187 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18694/tid=18701] ppid=18692 vsize=939380 CPUtime=4389.04 cores=0-7
/proc/18694/task/18701/stat : 18701 (claspmt) R 18692 18694 18668 0 -1 4202560 15180 0 0 0 438878 26 0 0 25 0 8 0 550267379 961925120 156697 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18694/tid=18702] ppid=18692 vsize=939380 CPUtime=4319.84 cores=0-7
/proc/18694/task/18702/stat : 18702 (claspmt) R 18692 18694 18668 0 -1 4202560 29487 0 0 0 431936 48 0 0 25 0 8 0 550267379 961925120 156697 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 34731.2
Current children cumulated vsize (KiB) 939380

[startup+5022.3 s]
/proc/loadavg: 9.07 9.04 9.00 10/293 19354
/proc/meminfo: memFree=4438032/32951124 swapFree=67051472/67111528
[pid=18694] ppid=18692 vsize=940872 CPUtime=35150.6 cores=0-7
/proc/18694/stat : 18694 (claspmt) R 18692 18694 18668 0 -1 4202496 171531 0 0 0 3514809 250 0 0 25 0 8 0 550267355 963452928 157825 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670219 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/18694/statm: 235218 157825 466 189 0 229567 0
[pid=18694/tid=18696] ppid=18692 vsize=940872 CPUtime=4178.49 cores=0-7
/proc/18694/task/18696/stat : 18696 (claspmt) R 18692 18694 18668 0 -1 4202560 29904 0 0 0 417815 34 0 0 25 0 8 0 550267379 963452928 157825 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18694/tid=18697] ppid=18692 vsize=940872 CPUtime=4268.39 cores=0-7
/proc/18694/task/18697/stat : 18697 (claspmt) R 18692 18694 18668 0 -1 4202560 30149 0 0 0 426809 30 0 0 25 0 8 0 550267379 963452928 157825 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18694/tid=18698] ppid=18692 vsize=940872 CPUtime=4428.42 cores=0-7
/proc/18694/task/18698/stat : 18698 (claspmt) R 18692 18694 18668 0 -1 4202560 14264 0 0 0 442815 27 0 0 25 0 8 0 550267379 963452928 157825 33554432000 4194304 4964704 140733977311968 18446744073709551615 4668714 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18694/tid=18699] ppid=18692 vsize=940872 CPUtime=4566.26 cores=0-7
/proc/18694/task/18699/stat : 18699 (claspmt) R 18692 18694 18668 0 -1 4202560 18095 0 0 0 456601 25 0 0 25 0 8 0 550267379 963452928 157825 33554432000 4194304 4964704 140733977311968 18446744073709551615 4668693 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18694/tid=18700] ppid=18692 vsize=940872 CPUtime=4394.27 cores=0-7
/proc/18694/task/18700/stat : 18700 (claspmt) R 18692 18694 18668 0 -1 4202560 17368 0 0 0 439407 20 0 0 25 0 8 0 550267379 963452928 157825 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18694/tid=18701] ppid=18692 vsize=940872 CPUtime=4429.4 cores=0-7
/proc/18694/task/18701/stat : 18701 (claspmt) R 18692 18694 18668 0 -1 4202560 15427 0 0 0 442913 27 0 0 25 0 8 0 550267379 963452928 157825 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18694/tid=18702] ppid=18692 vsize=940872 CPUtime=4379.85 cores=0-7
/proc/18694/task/18702/stat : 18702 (claspmt) R 18692 18694 18668 0 -1 4202560 29727 0 0 0 437936 49 0 0 25 0 8 0 550267379 963452928 157825 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 35150.6
Current children cumulated vsize (KiB) 940872

[startup+5082.3 s]
/proc/loadavg: 9.02 9.03 9.00 10/293 19356
/proc/meminfo: memFree=4437808/32951124 swapFree=67051472/67111528
[pid=18694] ppid=18692 vsize=941156 CPUtime=35570 cores=0-7
/proc/18694/stat : 18694 (claspmt) R 18692 18694 18668 0 -1 4202496 172023 0 0 0 3556750 252 0 0 25 0 8 0 550267355 963743744 158189 33554432000 4194304 4964704 140733977311968 18446744073709551615 4321069 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/18694/statm: 235289 158189 466 189 0 229638 0
[pid=18694/tid=18696] ppid=18692 vsize=941156 CPUtime=4224.8 cores=0-7
/proc/18694/task/18696/stat : 18696 (claspmt) R 18692 18694 18668 0 -1 4202560 30064 0 0 0 422446 34 0 0 25 0 8 0 550267379 963743744 158189 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670427 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18694/tid=18697] ppid=18692 vsize=941156 CPUtime=4309.55 cores=0-7
/proc/18694/task/18697/stat : 18697 (claspmt) R 18692 18694 18668 0 -1 4202560 30253 0 0 0 430924 31 0 0 25 0 8 0 550267379 963743744 158189 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18694/tid=18698] ppid=18692 vsize=941156 CPUtime=4471.31 cores=0-7
/proc/18694/task/18698/stat : 18698 (claspmt) R 18692 18694 18668 0 -1 4202560 14320 0 0 0 447104 27 0 0 25 0 8 0 550267379 963743744 158189 33554432000 4194304 4964704 140733977311968 18446744073709551615 4659143 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18694/tid=18699] ppid=18692 vsize=941156 CPUtime=4621.18 cores=0-7
/proc/18694/task/18699/stat : 18699 (claspmt) R 18692 18694 18668 0 -1 4202560 18123 0 0 0 462093 25 0 0 25 0 8 0 550267379 963743744 158189 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18694/tid=18700] ppid=18692 vsize=941156 CPUtime=4454.26 cores=0-7
/proc/18694/task/18700/stat : 18700 (claspmt) R 18692 18694 18668 0 -1 4202560 17368 0 0 0 445406 20 0 0 25 0 8 0 550267379 963743744 158189 33554432000 4194304 4964704 140733977311968 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18694/tid=18701] ppid=18692 vsize=941156 CPUtime=4483.55 cores=0-7
/proc/18694/task/18701/stat : 18701 (claspmt) R 18692 18694 18668 0 -1 4202560 15491 0 0 0 448328 27 0 0 25 0 8 0 550267379 963743744 158189 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18694/tid=18702] ppid=18692 vsize=941156 CPUtime=4439.84 cores=0-7
/proc/18694/task/18702/stat : 18702 (claspmt) R 18692 18694 18668 0 -1 4202560 29807 0 0 0 443935 49 0 0 25 0 8 0 550267379 963743744 158189 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 35570
Current children cumulated vsize (KiB) 941156



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 9.02 9.03 9.00 10/293 19357
/proc/meminfo: memFree=4437764/32951124 swapFree=67051472/67111528
[pid=18694] ppid=18692 vsize=941156 CPUtime=35693.8 cores=0-7
/proc/18694/stat : 18694 (claspmt) R 18692 18694 18668 0 -1 4202496 172072 0 0 0 3569129 253 0 0 25 0 8 0 550267355 963743744 158238 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670219 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/18694/statm: 235289 158238 466 189 0 229638 0
[pid=18694/tid=18696] ppid=18692 vsize=941156 CPUtime=4233.68 cores=0-7
/proc/18694/task/18696/stat : 18696 (claspmt) R 18692 18694 18668 0 -1 4202560 30064 0 0 0 423334 34 0 0 25 0 8 0 550267379 963743744 158238 33554432000 4194304 4964704 140733977311968 18446744073709551615 4668668 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18694/tid=18697] ppid=18692 vsize=941156 CPUtime=4318.39 cores=0-7
/proc/18694/task/18697/stat : 18697 (claspmt) R 18692 18694 18668 0 -1 4202560 30266 0 0 0 431808 31 0 0 25 0 8 0 550267379 963743744 158238 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=18694/tid=18698] ppid=18692 vsize=941156 CPUtime=4489.02 cores=0-7
/proc/18694/task/18698/stat : 18698 (claspmt) R 18692 18694 18668 0 -1 4202560 14320 0 0 0 448875 27 0 0 25 0 8 0 550267379 963743744 158238 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670263 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=18694/tid=18699] ppid=18692 vsize=941156 CPUtime=4638.89 cores=0-7
/proc/18694/task/18699/stat : 18699 (claspmt) R 18692 18694 18668 0 -1 4202560 18123 0 0 0 463864 25 0 0 25 0 8 0 550267379 963743744 158238 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18694/tid=18700] ppid=18692 vsize=941156 CPUtime=4471.97 cores=0-7
/proc/18694/task/18700/stat : 18700 (claspmt) R 18692 18694 18668 0 -1 4202560 17404 0 0 0 447177 20 0 0 25 0 8 0 550267379 963743744 158238 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=18694/tid=18701] ppid=18692 vsize=941156 CPUtime=4501.09 cores=0-7
/proc/18694/task/18701/stat : 18701 (claspmt) R 18692 18694 18668 0 -1 4202560 15491 0 0 0 450082 27 0 0 25 0 8 0 550267379 963743744 158238 33554432000 4194304 4964704 140733977311968 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=18694/tid=18702] ppid=18692 vsize=941156 CPUtime=4457.55 cores=0-7
/proc/18694/task/18702/stat : 18702 (claspmt) R 18692 18694 18668 0 -1 4202560 29807 0 0 0 445706 49 0 0 25 0 8 0 550267379 963743744 158238 33554432000 4194304 4964704 140733977311968 18446744073709551615 218370616052 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 35693.8
Current children cumulated vsize (KiB) 941156

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5100.12 s]
/proc/loadavg: 9.02 9.03 9.00 10/293 19357
/proc/meminfo: memFree=4437764/32951124 swapFree=67051472/67111528
[pid=18694] ppid=18692 vsize=932220 CPUtime=35694 cores=0-7
/proc/18694/stat : 18694 (claspmt) R 18692 18694 18668 0 -1 4202496 172082 0 0 0 3569143 253 0 0 25 0 1 0 550267355 954593280 155254 33554432000 4194304 4964704 140733977311968 18446744073709551615 4646575 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/18694/statm: 233055 155254 476 189 0 227404 0
Current children cumulated CPU time (s) 35694
Current children cumulated vsize (KiB) 932220

Child status: 0
Real time (s): 5100.17
CPU time (s): 35694
CPU user time (s): 35691.5
CPU system time (s): 2.56361
CPU usage (%): 699.859
Max. virtual memory (cumulated for all children) (KiB): 941156

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 35691.5
system time used= 2.56361
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 172083
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 129232
involuntary context switches= 239132

runsolver used 12.7701 second user time and 33.6219 second system time

The end

Launcher Data 

Begin job on node101 at 2011-05-03 03:37:26
IDJOB=3322486
IDBENCH=83321
IDSOLVER=1711
FILE ID=node101/3322486-1304386646
RUNJOBID= node101-1304386646-18682
PBS_JOBID= 13181808
Free space on /tmp= 73076 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT11/crafted/kullmann/VanDerWaerden/VanderWaerden3k/VanDerWaerden_2-3-20_389.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3322486-1304386646/watcher-3322486-1304386646 -o /tmp/evaluation-result-3322486-1304386646/solver-3322486-1304386646 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322486-1304386646.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= f105570f47bdca662c2d3d185f8dc1f8
RANDOM SEED=2010467090

node101.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:       5071680 kB
Buffers:        262756 kB
Cached:        2398780 kB
SwapCached:      23472 kB
Active:       25277036 kB
Inactive:      2252808 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:       5071680 kB
SwapTotal:    67111528 kB
SwapFree:     67051472 kB
Dirty:            2020 kB
Writeback:           0 kB
AnonPages:    24867572 kB
Mapped:          22564 kB
Slab:           222240 kB
PageTables:      63376 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 42437280 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73480 MiB
End job on node101 at 2011-05-03 05:02:28