Trace number 3741350

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
PB10: pb_cplex 2010-06-29? (TO) 1800.39 515.617

General information on the benchmark

Name/OPT-SMALLINT-LIN/heinz/
normalized-p6b.opb
MD5SUMf076140978d89a37e97e041a8b1e6b8f
Bench CategoryOPT-SMALLINT-LIN (optimisation, small integers, linear constraints)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark-63
Best CPU time to get the best result obtained on this benchmark1796.78
Has Objective FunctionYES
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables462
Total number of constraints5852
Number of constraints which are clauses5852
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint2
Maximum length of a constraint2
Number of terms in the objective function 462
Biggest coefficient in the objective function 1
Number of bits for the biggest coefficient in the objective function 1
Sum of the numbers in the objective function 462
Number of bits of the sum of numbers in the objective function 9
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 462
Number of bits of the biggest sum of numbers9
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing opb file format. File HOME/instance-3741350-1339250186.opb.
0.00/0.00	c Instance file HOME/instance-3741350-1339250186.opb
0.00/0.00	c File size is 141885 bytes.
0.00/0.03	c Highest Coefficient sum: 3
0.00/0.03	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
0.00/0.04	c ============================[ Problem Statistics ]=============================
0.00/0.04	c |                                                                             |
0.00/0.04	c |  Parsing time:          0.04         s                                      |
0.00/0.04	c |  Number Variables:      462                                                 |
0.00/0.04	c |  Number Constraints:    5852                                                |
0.00/0.04	c ===============================================================================
1799.90/515.61	c Remaining time 1799.96 sec.
1799.90/515.61	c 
1799.90/515.61	c *** INTERRUPTED ***
1799.90/515.61	s UNKNOWN
1799.90/515.61	c Total CPU time              : 1664.23 s
1799.90/515.61	c Memory used           : 228.59 MB
1799.90/515.61	c CPU time              : 1664.23 s
1799.90/515.61	c 
1799.90/515.61	c *** INTERRUPTED ***

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3741350-1339250186/watcher-3741350-1339250186 -o /tmp/evaluation-result-3741350-1339250186/solver-3741350-1339250186 -C 1800 -W 1900 -M 15500 run HOME/instance-3741350-1339250186.opb 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.05 4.84 4.80 3/176 30616
/proc/meminfo: memFree=29238508/32873844 swapFree=7628/7628
[pid=30616] ppid=30614 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/30616/stat : 30616 (run) S 30614 30616 29579 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 391142980 9367552 270 33554432000 4194304 5064252 140737436314000 140737436312912 251044456014 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/30616/statm: 2287 270 229 213 0 72 0

[startup+0.0827661 s]
/proc/loadavg: 6.05 4.84 4.80 3/176 30616
/proc/meminfo: memFree=29238508/32873844 swapFree=7628/7628
[pid=30616] ppid=30614 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/30616/stat : 30616 (run) S 30614 30616 29579 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 391142980 9367552 270 33554432000 4194304 5064252 140737436314000 140737436312912 251044456014 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/30616/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.100251 s]
/proc/loadavg: 6.05 4.84 4.80 3/176 30616
/proc/meminfo: memFree=29238508/32873844 swapFree=7628/7628
[pid=30616] ppid=30614 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/30616/stat : 30616 (run) S 30614 30616 29579 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 391142980 9367552 270 33554432000 4194304 5064252 140737436314000 140737436312912 251044456014 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/30616/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.300182 s]
/proc/loadavg: 6.05 4.84 4.80 3/176 30616
/proc/meminfo: memFree=29238508/32873844 swapFree=7628/7628
[pid=30616] ppid=30614 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/30616/stat : 30616 (run) S 30614 30616 29579 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 391142980 9367552 270 33554432000 4194304 5064252 140737436314000 140737436312912 251044456014 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/30616/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.700181 s]
/proc/loadavg: 6.05 4.84 4.80 3/176 30616
/proc/meminfo: memFree=29238508/32873844 swapFree=7628/7628
[pid=30616] ppid=30614 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/30616/stat : 30616 (run) S 30614 30616 29579 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 391142980 9367552 270 33554432000 4194304 5064252 140737436314000 140737436312912 251044456014 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/30616/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+1.50041 s]
/proc/loadavg: 6.05 4.84 4.80 3/185 30625
/proc/meminfo: memFree=29226912/32873844 swapFree=7628/7628
[pid=30616] ppid=30614 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/30616/stat : 30616 (run) S 30614 30616 29579 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 391142980 9367552 270 33554432000 4194304 5064252 140737436314000 140737436312912 251044456014 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/30616/statm: 2287 270 229 213 0 72 0
[pid=30617] ppid=30616 vsize=109204 CPUtime=2.06 cores=1,3,5,7
/proc/30617/stat : 30617 (pb_cplex) R 30616 30616 29579 0 -1 4202496 7421 0 0 0 204 2 0 0 20 0 8 0 391142980 111824896 5753 33554432000 134512640 146736320 4287600480 4287593360 136008050 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/30617/statm: 27301 5753 934 2985 0 24235 0
[pid=30617/tid=30619] ppid=30616 vsize=109204 CPUtime=0.05 cores=1,3,5,7
/proc/30617/task/30619/stat : 30619 (pb_cplex) R 30616 30616 29579 0 -1 4202560 227 0 0 0 5 0 0 0 20 0 8 0 391143079 111824896 5753 33554432000 134512640 146736320 4287600480 4151809000 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30620] ppid=30616 vsize=109204 CPUtime=0.04 cores=1,3,5,7
/proc/30617/task/30620/stat : 30620 (pb_cplex) R 30616 30616 29579 0 -1 4202560 282 0 0 0 4 0 0 0 20 0 8 0 391143079 111824896 5753 33554432000 134512640 146736320 4287600480 4141322696 136001717 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=30617/tid=30621] ppid=30616 vsize=109204 CPUtime=0.12 cores=1,3,5,7
/proc/30617/task/30621/stat : 30621 (pb_cplex) R 30616 30616 29579 0 -1 4202560 275 0 0 0 12 0 0 0 20 0 8 0 391143079 111824896 5753 33554432000 134512640 146736320 4287600480 4130837480 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=30617/tid=30622] ppid=30616 vsize=109204 CPUtime=0.11 cores=1,3,5,7
/proc/30617/task/30622/stat : 30622 (pb_cplex) R 30616 30616 29579 0 -1 4202560 294 0 0 0 11 0 0 0 20 0 8 0 391143079 111824896 5753 33554432000 134512640 146736320 4287600480 4120350776 136475862 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=30617/tid=30623] ppid=30616 vsize=109204 CPUtime=0.05 cores=1,3,5,7
/proc/30617/task/30623/stat : 30623 (pb_cplex) R 30616 30616 29579 0 -1 4202560 237 0 0 0 5 0 0 0 20 0 8 0 391143079 111824896 5753 33554432000 134512640 146736320 4287600480 4109865052 135855127 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=30617/tid=30624] ppid=30616 vsize=109204 CPUtime=0.14 cores=1,3,5,7
/proc/30617/task/30624/stat : 30624 (pb_cplex) R 30616 30616 29579 0 -1 4202560 239 0 0 0 14 0 0 0 20 0 8 0 391143079 111824896 5753 33554432000 134512640 146736320 4287600480 4099379420 136009316 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30625] ppid=30616 vsize=109204 CPUtime=0.06 cores=1,3,5,7
/proc/30617/task/30625/stat : 30625 (pb_cplex) R 30616 30616 29579 0 -1 4202560 261 0 0 0 6 0 0 0 20 0 8 0 391143079 111824896 5753 33554432000 134512640 146736320 4287600480 4088893508 136385174 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 2.06
Current children cumulated vsize (KiB) 118352

[startup+3.10036 s]
/proc/loadavg: 6.05 4.84 4.80 10/185 30625
/proc/meminfo: memFree=29219348/32873844 swapFree=7628/7628
[pid=30616] ppid=30614 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/30616/stat : 30616 (run) S 30614 30616 29579 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 391142980 9367552 270 33554432000 4194304 5064252 140737436314000 140737436312912 251044456014 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/30616/statm: 2287 270 229 213 0 72 0
[pid=30617] ppid=30616 vsize=109468 CPUtime=7.89 cores=1,3,5,7
/proc/30617/stat : 30617 (pb_cplex) R 30616 30616 29579 0 -1 4202496 7574 0 0 0 784 5 0 0 20 0 8 0 391142980 112095232 5889 33554432000 134512640 146736320 4287600480 4287594152 144757825 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/30617/statm: 27367 5889 935 2985 0 24301 0
[pid=30617/tid=30619] ppid=30616 vsize=109468 CPUtime=0.94 cores=1,3,5,7
/proc/30617/task/30619/stat : 30619 (pb_cplex) R 30616 30616 29579 0 -1 4202560 227 0 0 0 94 0 0 0 20 0 8 0 391143079 112095232 5889 33554432000 134512640 146736320 4287600480 4151808416 136388288 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=30617/tid=30620] ppid=30616 vsize=109468 CPUtime=0.8 cores=1,3,5,7
/proc/30617/task/30620/stat : 30620 (pb_cplex) R 30616 30616 29579 0 -1 4202560 283 0 0 0 80 0 0 0 20 0 8 0 391143079 112095232 5889 33554432000 134512640 146736320 4287600480 4141322632 136388081 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30621] ppid=30616 vsize=109468 CPUtime=0.93 cores=1,3,5,7
/proc/30617/task/30621/stat : 30621 (pb_cplex) R 30616 30616 29579 0 -1 4202560 308 0 0 0 93 0 0 0 20 0 8 0 391143079 112095232 5889 33554432000 134512640 146736320 4287600480 4130835600 136008065 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=30617/tid=30622] ppid=30616 vsize=109468 CPUtime=0.75 cores=1,3,5,7
/proc/30617/task/30622/stat : 30622 (pb_cplex) R 30616 30616 29579 0 -1 4202560 357 0 0 0 75 0 0 0 20 0 8 0 391143079 112095232 5889 33554432000 134512640 146736320 4287600480 4120350760 139044585 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=30617/tid=30623] ppid=30616 vsize=109468 CPUtime=0.77 cores=1,3,5,7
/proc/30617/task/30623/stat : 30623 (pb_cplex) R 30616 30616 29579 0 -1 4202560 237 0 0 0 77 0 0 0 20 0 8 0 391143079 112095232 5889 33554432000 134512640 146736320 4287600480 4109865960 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=30617/tid=30624] ppid=30616 vsize=109468 CPUtime=0.75 cores=1,3,5,7
/proc/30617/task/30624/stat : 30624 (pb_cplex) R 30616 30616 29579 0 -1 4202560 295 0 0 0 75 0 0 0 20 0 8 0 391143079 112095232 5889 33554432000 134512640 146736320 4287600480 4099379408 136007970 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30625] ppid=30616 vsize=109468 CPUtime=0.73 cores=1,3,5,7
/proc/30617/task/30625/stat : 30625 (pb_cplex) R 30616 30616 29579 0 -1 4202560 261 0 0 0 73 0 0 0 20 0 8 0 391143079 112095232 5889 33554432000 134512640 146736320 4287600480 4088891112 138462623 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 7.89
Current children cumulated vsize (KiB) 118616

[startup+6.30024 s]
/proc/loadavg: 6.13 4.88 4.81 3/179 30626
/proc/meminfo: memFree=29219892/32873844 swapFree=7628/7628
[pid=30616] ppid=30614 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/30616/stat : 30616 (run) S 30614 30616 29579 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 391142980 9367552 270 33554432000 4194304 5064252 140737436314000 140737436312912 251044456014 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/30616/statm: 2287 270 229 213 0 72 0
[pid=30617] ppid=30616 vsize=161068 CPUtime=13.88 cores=1,3,5,7
/proc/30617/stat : 30617 (pb_cplex) R 30616 30616 29579 0 -1 4202496 16074 0 0 0 1366 22 0 0 20 0 8 0 391142980 164933632 15674 33554432000 134512640 146736320 4287600480 4287592704 139048112 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0 0 0
/proc/30617/statm: 40267 15674 947 2985 0 37201 0
Current children cumulated CPU time (s) 13.88
Current children cumulated vsize (KiB) 170216

[startup+12.7004 s]
/proc/loadavg: 5.80 4.83 4.80 5/186 30640
/proc/meminfo: memFree=29167068/32873844 swapFree=7628/7628
[pid=30616] ppid=30614 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/30616/stat : 30616 (run) S 30614 30616 29579 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 391142980 9367552 270 33554432000 4194304 5064252 140737436314000 140737436312912 251044456014 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/30616/statm: 2287 270 229 213 0 72 0
[pid=30617] ppid=30616 vsize=170288 CPUtime=34.38 cores=1,3,5,7
/proc/30617/stat : 30617 (pb_cplex) S 30616 30616 29579 0 -1 4202496 22589 0 0 0 3403 35 0 0 20 0 8 0 391142980 174374912 17976 33554432000 134512640 146736320 4287600480 4287596880 144580073 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/30617/statm: 42572 17976 952 2985 0 39506 0
[pid=30617/tid=30634] ppid=30616 vsize=170288 CPUtime=2.58 cores=1,3,5,7
/proc/30617/task/30634/stat : 30634 (pb_cplex) R 30616 30616 29579 0 -1 4202560 818 0 0 0 256 2 0 0 20 0 8 0 391143599 174374912 17976 33554432000 134512640 146736320 4287600480 4070563752 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30635] ppid=30616 vsize=170288 CPUtime=2.39 cores=1,3,5,7
/proc/30617/task/30635/stat : 30635 (pb_cplex) R 30616 30616 29579 0 -1 4202560 1114 0 0 0 238 1 0 0 20 0 8 0 391143599 174374912 17976 33554432000 134512640 146736320 4287600480 4131470548 136003851 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30636] ppid=30616 vsize=170288 CPUtime=2.16 cores=1,3,5,7
/proc/30617/task/30636/stat : 30636 (pb_cplex) R 30616 30616 29579 0 -1 4202560 714 0 0 0 215 1 0 0 20 0 8 0 391143599 174374912 17976 33554432000 134512640 146736320 4287600480 4141957032 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=30617/tid=30637] ppid=30616 vsize=170288 CPUtime=2.49 cores=1,3,5,7
/proc/30617/task/30637/stat : 30637 (pb_cplex) R 30616 30616 29579 0 -1 4202560 1731 0 0 0 248 1 0 0 20 0 8 0 391143599 174374912 17976 33554432000 134512640 146736320 4287600480 4120350812 135855162 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=30617/tid=30638] ppid=30616 vsize=170288 CPUtime=2.36 cores=1,3,5,7
/proc/30617/task/30638/stat : 30638 (pb_cplex) R 30616 30616 29579 0 -1 4202560 984 0 0 0 234 2 0 0 20 0 8 0 391143599 174374912 17976 33554432000 134512640 146736320 4287600480 4040367248 136008065 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=30617/tid=30639] ppid=30616 vsize=170288 CPUtime=2.37 cores=1,3,5,7
/proc/30617/task/30639/stat : 30639 (pb_cplex) R 30616 30616 29579 0 -1 4202560 477 0 0 0 236 1 0 0 20 0 8 0 391143599 174374912 17976 33554432000 134512640 146736320 4287600480 4029882460 135855174 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=30617/tid=30640] ppid=30616 vsize=170288 CPUtime=2.05 cores=1,3,5,7
/proc/30617/task/30640/stat : 30640 (pb_cplex) R 30616 30616 29579 0 -1 4202560 655 0 0 0 205 0 0 0 20 0 8 0 391143599 174374912 17976 33554432000 134512640 146736320 4287600480 4019395336 135525920 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
Current children cumulated CPU time (s) 34.38
Current children cumulated vsize (KiB) 179436

[startup+25.5004 s]
/proc/loadavg: 6.42 5.02 4.86 3/186 30640
/proc/meminfo: memFree=29051844/32873844 swapFree=7628/7628
[pid=30616] ppid=30614 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/30616/stat : 30616 (run) S 30614 30616 29579 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 391142980 9367552 270 33554432000 4194304 5064252 140737436314000 140737436312912 251044456014 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/30616/statm: 2287 270 229 213 0 72 0
[pid=30617] ppid=30616 vsize=173884 CPUtime=77.87 cores=1,3,5,7
/proc/30617/stat : 30617 (pb_cplex) R 30616 30616 29579 0 -1 4202496 27041 0 0 0 7701 86 0 0 20 0 8 0 391142980 178057216 18539 33554432000 134512640 146736320 4287600480 4287593216 136382629 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/30617/statm: 43471 18539 954 2985 0 40405 0
[pid=30617/tid=30634] ppid=30616 vsize=173884 CPUtime=7.84 cores=1,3,5,7
/proc/30617/task/30634/stat : 30634 (pb_cplex) S 30616 30616 29579 0 -1 4202560 1437 0 0 0 772 12 0 0 20 0 8 0 391143599 178057216 18539 33554432000 134512640 146736320 4287600480 4070567568 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30635] ppid=30616 vsize=173884 CPUtime=8.25 cores=1,3,5,7
/proc/30617/task/30635/stat : 30635 (pb_cplex) R 30616 30616 29579 0 -1 4202560 1253 0 0 0 819 6 0 0 20 0 8 0 391143599 178057216 18539 33554432000 134512640 146736320 4287600480 4131471452 135855165 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=30617/tid=30636] ppid=30616 vsize=173884 CPUtime=7.31 cores=1,3,5,7
/proc/30617/task/30636/stat : 30636 (pb_cplex) R 30616 30616 29579 0 -1 4202560 1226 0 0 0 723 8 0 0 20 0 8 0 391143599 178057216 18539 33554432000 134512640 146736320 4287600480 4141958120 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30637] ppid=30616 vsize=173884 CPUtime=7.54 cores=1,3,5,7
/proc/30617/task/30637/stat : 30637 (pb_cplex) R 30616 30616 29579 0 -1 4202560 2733 0 0 0 748 6 0 0 20 0 8 0 391143599 178057216 18539 33554432000 134512640 146736320 4287600480 4120351176 136001700 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=30617/tid=30638] ppid=30616 vsize=173884 CPUtime=7.71 cores=1,3,5,7
/proc/30617/task/30638/stat : 30638 (pb_cplex) S 30616 30616 29579 0 -1 4202560 1574 0 0 0 765 6 0 0 20 0 8 0 391143599 178057216 18539 33554432000 134512640 146736320 4287600480 4040371856 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=30617/tid=30639] ppid=30616 vsize=173884 CPUtime=7.95 cores=1,3,5,7
/proc/30617/task/30639/stat : 30639 (pb_cplex) R 30616 30616 29579 0 -1 4202560 867 0 0 0 789 6 0 0 20 0 8 0 391143599 178057216 18539 33554432000 134512640 146736320 4287600480 4029883368 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=30617/tid=30640] ppid=30616 vsize=173884 CPUtime=7.72 cores=1,3,5,7
/proc/30617/task/30640/stat : 30640 (pb_cplex) R 30616 30616 29579 0 -1 4202560 1472 0 0 0 764 8 0 0 20 0 8 0 391143599 178057216 18539 33554432000 134512640 146736320 4287600480 4019396680 139042902 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 77.87
Current children cumulated vsize (KiB) 183032

[startup+51.1004 s]
/proc/loadavg: 6.77 5.21 4.93 10/186 30640
/proc/meminfo: memFree=29000164/32873844 swapFree=7628/7628
[pid=30616] ppid=30614 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/30616/stat : 30616 (run) S 30614 30616 29579 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 391142980 9367552 270 33554432000 4194304 5064252 140737436314000 140737436312912 251044456014 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/30616/statm: 2287 270 229 213 0 72 0
[pid=30617] ppid=30616 vsize=190924 CPUtime=167.55 cores=1,3,5,7
/proc/30617/stat : 30617 (pb_cplex) R 30616 30616 29579 0 -1 4202496 38699 0 0 0 16141 614 0 0 20 0 8 0 391142980 195506176 24345 33554432000 134512640 146736320 4287600480 4287588928 136018931 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/30617/statm: 47731 24345 955 2985 0 44665 0
[pid=30617/tid=30634] ppid=30616 vsize=190924 CPUtime=19.28 cores=1,3,5,7
/proc/30617/task/30634/stat : 30634 (pb_cplex) R 30616 30616 29579 0 -1 4202560 2327 0 0 0 1865 63 0 0 20 0 8 0 391143599 195506176 24345 33554432000 134512640 146736320 4287600480 4070559824 136008050 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30635] ppid=30616 vsize=190924 CPUtime=19.37 cores=1,3,5,7
/proc/30617/task/30635/stat : 30635 (pb_cplex) R 30616 30616 29579 0 -1 4202560 2083 0 0 0 1867 70 0 0 20 0 8 0 391143599 195506176 24345 33554432000 134512640 146736320 4287600480 4131467592 136000758 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30636] ppid=30616 vsize=190924 CPUtime=18.09 cores=1,3,5,7
/proc/30617/task/30636/stat : 30636 (pb_cplex) R 30616 30616 29579 0 -1 4202560 2787 0 0 0 1748 61 0 0 20 0 8 0 391143599 195506176 24345 33554432000 134512640 146736320 4287600480 4141953104 136008042 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=30617/tid=30637] ppid=30616 vsize=190924 CPUtime=19.04 cores=1,3,5,7
/proc/30617/task/30637/stat : 30637 (pb_cplex) R 30616 30616 29579 0 -1 4202560 6204 0 0 0 1838 66 0 0 20 0 8 0 391143599 195506176 24345 33554432000 134512640 146736320 4287600480 4120345684 136004196 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=30617/tid=30638] ppid=30616 vsize=190924 CPUtime=18.65 cores=1,3,5,7
/proc/30617/task/30638/stat : 30638 (pb_cplex) R 30616 30616 29579 0 -1 4202560 2651 0 0 0 1794 71 0 0 20 0 8 0 391143599 195506176 24345 33554432000 134512640 146736320 4287600480 4040364908 144785853 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=30617/tid=30639] ppid=30616 vsize=190924 CPUtime=19.12 cores=1,3,5,7

################
# More data... #
################

[pid=30617/tid=30645] ppid=30616 vsize=194864 CPUtime=63.53 cores=1,3,5,7
/proc/30617/task/30645/stat : 30645 (pb_cplex) R 30616 30616 29579 0 -1 4202560 845 0 0 0 5810 543 0 0 20 0 8 0 391150568 199540736 26904 33554432000 134512640 146736320 4287600480 4070564840 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=30617/tid=30646] ppid=30616 vsize=194864 CPUtime=64.05 cores=1,3,5,7
/proc/30617/task/30646/stat : 30646 (pb_cplex) R 30616 30616 29579 0 -1 4202560 543 0 0 0 5886 519 0 0 20 0 8 0 391150568 199540736 26904 33554432000 134512640 146736320 4287600480 4060078172 135855165 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=30617/tid=30647] ppid=30616 vsize=194864 CPUtime=63.55 cores=1,3,5,7
/proc/30617/task/30647/stat : 30647 (pb_cplex) R 30616 30616 29579 0 -1 4202560 841 0 0 0 5878 477 0 0 20 0 8 0 391150568 199540736 26904 33554432000 134512640 146736320 4287600480 3985630232 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 767.99
Current children cumulated vsize (KiB) 204012

[startup+282.3 s]
/proc/loadavg: 6.20 6.04 5.34 10/187 30673
/proc/meminfo: memFree=28874916/32873844 swapFree=7628/7628
[pid=30616] ppid=30614 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/30616/stat : 30616 (run) S 30614 30616 29579 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 391142980 9367552 270 33554432000 4194304 5064252 140737436314000 140737436312912 251044456014 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/30616/statm: 2287 270 229 213 0 72 0
[pid=30617] ppid=30616 vsize=201724 CPUtime=981.43 cores=1,3,5,7
/proc/30617/stat : 30617 (pb_cplex) R 30616 30616 29579 0 -1 4202496 88066 0 0 0 91483 6660 0 0 20 0 8 0 391142980 206565376 28522 33554432000 134512640 146736320 4287600480 4287593220 136385170 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0 0 0
/proc/30617/statm: 50431 28522 981 2985 0 47365 0
[pid=30617/tid=30641] ppid=30616 vsize=201724 CPUtime=90.86 cores=1,3,5,7
/proc/30617/task/30641/stat : 30641 (pb_cplex) R 30616 30616 29579 0 -1 4202560 810 0 0 0 8351 735 0 0 20 0 8 0 391150568 206565376 28522 33554432000 134512640 146736320 4287600480 4019397608 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=30617/tid=30642] ppid=30616 vsize=201724 CPUtime=90.92 cores=1,3,5,7
/proc/30617/task/30642/stat : 30642 (pb_cplex) R 30616 30616 29579 0 -1 4202560 2398 0 0 0 8312 780 0 0 20 0 8 0 391150568 206565376 28522 33554432000 134512640 146736320 4287600480 4029883368 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=30617/tid=30643] ppid=30616 vsize=201724 CPUtime=90.45 cores=1,3,5,7
/proc/30617/task/30643/stat : 30643 (pb_cplex) R 30616 30616 29579 0 -1 4202560 2227 0 0 0 8292 753 0 0 20 0 8 0 391150568 206565376 28522 33554432000 134512640 146736320 4287600480 4040368320 136469083 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30644] ppid=30616 vsize=201724 CPUtime=90.5 cores=1,3,5,7
/proc/30617/task/30644/stat : 30644 (pb_cplex) R 30616 30616 29579 0 -1 4202560 1134 0 0 0 8234 816 0 0 20 0 8 0 391150568 206565376 28522 33554432000 134512640 146736320 4287600480 4120354448 144580073 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30645] ppid=30616 vsize=201724 CPUtime=90.04 cores=1,3,5,7
/proc/30617/task/30645/stat : 30645 (pb_cplex) S 30616 30616 29579 0 -1 4202560 1364 0 0 0 8268 736 0 0 20 0 8 0 391150568 206565376 28522 33554432000 134512640 146736320 4287600480 4070567568 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=30617/tid=30646] ppid=30616 vsize=201724 CPUtime=91.07 cores=1,3,5,7
/proc/30617/task/30646/stat : 30646 (pb_cplex) S 30616 30616 29579 0 -1 4202560 1217 0 0 0 8359 748 0 0 20 0 8 0 391150568 206565376 28522 33554432000 134512640 146736320 4287600480 4060081808 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=30617/tid=30647] ppid=30616 vsize=201724 CPUtime=89.92 cores=1,3,5,7
/proc/30617/task/30647/stat : 30647 (pb_cplex) S 30616 30616 29579 0 -1 4202560 2712 0 0 0 8314 678 0 0 20 0 8 0 391150568 206565376 28522 33554432000 134512640 146736320 4287600480 3985632912 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 981.43
Current children cumulated vsize (KiB) 210872

[startup+342.3 s]
/proc/loadavg: 6.52 6.19 5.44 10/186 30674
/proc/meminfo: memFree=28866316/32873844 swapFree=7628/7628
[pid=30616] ppid=30614 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/30616/stat : 30616 (run) S 30614 30616 29579 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 391142980 9367552 270 33554432000 4194304 5064252 140737436314000 140737436312912 251044456014 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/30616/statm: 2287 270 229 213 0 72 0
[pid=30617] ppid=30616 vsize=211292 CPUtime=1193.25 cores=1,3,5,7
/proc/30617/stat : 30617 (pb_cplex) S 30616 30616 29579 0 -1 4202496 94523 0 0 0 110882 8443 0 0 20 0 8 0 391142980 216363008 30766 33554432000 134512640 146736320 4287600480 4287596880 144580073 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/30617/statm: 52823 30766 981 2985 0 49757 0
[pid=30617/tid=30641] ppid=30616 vsize=211292 CPUtime=117.29 cores=1,3,5,7
/proc/30617/task/30641/stat : 30641 (pb_cplex) R 30616 30616 29579 0 -1 4202560 1538 0 0 0 10782 947 0 0 20 0 8 0 391150568 216363008 30766 33554432000 134512640 146736320 4287600480 4019397608 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30642] ppid=30616 vsize=211292 CPUtime=117.07 cores=1,3,5,7
/proc/30617/task/30642/stat : 30642 (pb_cplex) R 30616 30616 29579 0 -1 4202560 3835 0 0 0 10672 1035 0 0 20 0 8 0 391150568 216363008 30766 33554432000 134512640 146736320 4287600480 4029882784 136388288 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=30617/tid=30643] ppid=30616 vsize=211292 CPUtime=116.49 cores=1,3,5,7
/proc/30617/task/30643/stat : 30643 (pb_cplex) R 30616 30616 29579 0 -1 4202560 2644 0 0 0 10676 973 0 0 20 0 8 0 391150568 216363008 30766 33554432000 134512640 146736320 4287600480 4040369176 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=30617/tid=30644] ppid=30616 vsize=211292 CPUtime=116.57 cores=1,3,5,7
/proc/30617/task/30644/stat : 30644 (pb_cplex) R 30616 30616 29579 0 -1 4202560 1791 0 0 0 10627 1030 0 0 20 0 8 0 391150568 216363008 30766 33554432000 134512640 146736320 4287600480 4120350928 136008335 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=30617/tid=30645] ppid=30616 vsize=211292 CPUtime=116.32 cores=1,3,5,7
/proc/30617/task/30645/stat : 30645 (pb_cplex) R 30616 30616 29579 0 -1 4202560 2281 0 0 0 10632 1000 0 0 20 0 8 0 391150568 216363008 30766 33554432000 134512640 146736320 4287600480 4070564048 136008053 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30646] ppid=30616 vsize=211292 CPUtime=117.59 cores=1,3,5,7
/proc/30617/task/30646/stat : 30646 (pb_cplex) R 30616 30616 29579 0 -1 4202560 1805 0 0 0 10823 936 0 0 20 0 8 0 391150568 216363008 30766 33554432000 134512640 146736320 4287600480 4060078148 136385179 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=30617/tid=30647] ppid=30616 vsize=211292 CPUtime=116.75 cores=1,3,5,7
/proc/30617/task/30647/stat : 30647 (pb_cplex) R 30616 30616 29579 0 -1 4202560 3894 0 0 0 10728 947 0 0 20 0 8 0 391150568 216363008 30766 33554432000 134512640 146736320 4287600480 3985630184 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1193.25
Current children cumulated vsize (KiB) 220440

[startup+402.3 s]
/proc/loadavg: 6.40 6.22 5.49 9/186 30674
/proc/meminfo: memFree=28859744/32873844 swapFree=7628/7628
[pid=30616] ppid=30614 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/30616/stat : 30616 (run) S 30614 30616 29579 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 391142980 9367552 270 33554432000 4194304 5064252 140737436314000 140737436312912 251044456014 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/30616/statm: 2287 270 229 213 0 72 0
[pid=30617] ppid=30616 vsize=219324 CPUtime=1402.7 cores=1,3,5,7
/proc/30617/stat : 30617 (pb_cplex) S 30616 30616 29579 0 -1 4202496 99671 0 0 0 130074 10196 0 0 20 0 8 0 391142980 224587776 32565 33554432000 134512640 146736320 4287600480 4287596880 144580073 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/30617/statm: 54831 32565 981 2985 0 51765 0
[pid=30617/tid=30641] ppid=30616 vsize=219324 CPUtime=143.73 cores=1,3,5,7
/proc/30617/task/30641/stat : 30641 (pb_cplex) S 30616 30616 29579 0 -1 4202560 1759 0 0 0 13232 1141 0 0 20 0 8 0 391150568 224587776 32565 33554432000 134512640 146736320 4287600480 4019400336 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=30617/tid=30642] ppid=30616 vsize=219324 CPUtime=143.41 cores=1,3,5,7
/proc/30617/task/30642/stat : 30642 (pb_cplex) S 30616 30616 29579 0 -1 4202560 4224 0 0 0 13099 1242 0 0 20 0 8 0 391150568 224587776 32565 33554432000 134512640 146736320 4287600480 4029886096 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=30617/tid=30643] ppid=30616 vsize=219324 CPUtime=142.45 cores=1,3,5,7
/proc/30617/task/30643/stat : 30643 (pb_cplex) S 30616 30616 29579 0 -1 4202560 2888 0 0 0 13078 1167 0 0 20 0 8 0 391150568 224587776 32565 33554432000 134512640 146736320 4287600480 4040371856 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=30617/tid=30644] ppid=30616 vsize=219324 CPUtime=142.5 cores=1,3,5,7
/proc/30617/task/30644/stat : 30644 (pb_cplex) S 30616 30616 29579 0 -1 4202560 2342 0 0 0 13022 1228 0 0 20 0 8 0 391150568 224587776 32565 33554432000 134512640 146736320 4287600480 4120354448 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=30617/tid=30645] ppid=30616 vsize=219324 CPUtime=141.97 cores=1,3,5,7
/proc/30617/task/30645/stat : 30645 (pb_cplex) S 30616 30616 29579 0 -1 4202560 3811 0 0 0 13006 1191 0 0 20 0 8 0 391150568 224587776 32565 33554432000 134512640 146736320 4287600480 4070567568 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30646] ppid=30616 vsize=219324 CPUtime=143.8 cores=1,3,5,7
/proc/30617/task/30646/stat : 30646 (pb_cplex) S 30616 30616 29579 0 -1 4202560 2256 0 0 0 13221 1159 0 0 20 0 8 0 391150568 224587776 32565 33554432000 134512640 146736320 4287600480 4060081808 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30647] ppid=30616 vsize=219324 CPUtime=143.29 cores=1,3,5,7
/proc/30617/task/30647/stat : 30647 (pb_cplex) R 30616 30616 29579 0 -1 4202560 4911 0 0 0 13164 1165 0 0 20 0 8 0 391150568 224587776 32565 33554432000 134512640 146736320 4287600480 3985629276 135855165 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
Current children cumulated CPU time (s) 1402.7
Current children cumulated vsize (KiB) 228472

[startup+462.3 s]
/proc/loadavg: 7.02 6.47 5.63 9/186 30674
/proc/meminfo: memFree=28850732/32873844 swapFree=7628/7628
[pid=30616] ppid=30614 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/30616/stat : 30616 (run) S 30614 30616 29579 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 391142980 9367552 270 33554432000 4194304 5064252 140737436314000 140737436312912 251044456014 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/30616/statm: 2287 270 229 213 0 72 0
[pid=30617] ppid=30616 vsize=223604 CPUtime=1612.92 cores=1,3,5,7
/proc/30617/stat : 30617 (pb_cplex) R 30616 30616 29579 0 -1 4202496 104706 0 0 0 149300 11992 0 0 20 0 8 0 391142980 228970496 34620 33554432000 134512640 146736320 4287600480 4287593428 136003826 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0 0 0
/proc/30617/statm: 55901 34620 981 2985 0 52835 0
[pid=30617/tid=30641] ppid=30616 vsize=223604 CPUtime=169.85 cores=1,3,5,7
/proc/30617/task/30641/stat : 30641 (pb_cplex) R 30616 30616 29579 0 -1 4202560 2394 0 0 0 15674 1311 0 0 20 0 8 0 391150568 228970496 34620 33554432000 134512640 146736320 4287600480 4019397472 135845837 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=30617/tid=30642] ppid=30616 vsize=223604 CPUtime=169.63 cores=1,3,5,7
/proc/30617/task/30642/stat : 30642 (pb_cplex) R 30616 30616 29579 0 -1 4202560 4587 0 0 0 15496 1467 0 0 20 0 8 0 391150568 228970496 34620 33554432000 134512640 146736320 4287600480 4029882368 136018931 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=30617/tid=30643] ppid=30616 vsize=223604 CPUtime=168.65 cores=1,3,5,7
/proc/30617/task/30643/stat : 30643 (pb_cplex) R 30616 30616 29579 0 -1 4202560 3297 0 0 0 15530 1335 0 0 20 0 8 0 391150568 228970496 34620 33554432000 134512640 146736320 4287600480 4040368348 136009749 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30644] ppid=30616 vsize=223604 CPUtime=168.66 cores=1,3,5,7
/proc/30617/task/30644/stat : 30644 (pb_cplex) R 30616 30616 29579 0 -1 4202560 2691 0 0 0 15414 1452 0 0 20 0 8 0 391150568 228970496 34620 33554432000 134512640 146736320 4287600480 4120354476 144580332 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=30617/tid=30645] ppid=30616 vsize=223604 CPUtime=168.24 cores=1,3,5,7
/proc/30617/task/30645/stat : 30645 (pb_cplex) R 30616 30616 29579 0 -1 4202560 4848 0 0 0 15427 1397 0 0 20 0 8 0 391150568 228970496 34620 33554432000 134512640 146736320 4287600480 4070564060 136389135 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=30617/tid=30646] ppid=30616 vsize=223604 CPUtime=170.11 cores=1,3,5,7
/proc/30617/task/30646/stat : 30646 (pb_cplex) R 30616 30616 29579 0 -1 4202560 3238 0 0 0 15634 1377 0 0 20 0 8 0 391150568 228970496 34620 33554432000 134512640 146736320 4287600480 4060079080 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30647] ppid=30616 vsize=223604 CPUtime=169.03 cores=1,3,5,7
/proc/30617/task/30647/stat : 30647 (pb_cplex) R 30616 30616 29579 0 -1 4202560 5870 0 0 0 15553 1350 0 0 20 0 8 0 391150568 228970496 34620 33554432000 134512640 146736320 4287600480 3985630184 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1612.92
Current children cumulated vsize (KiB) 232752



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+515.6 s]
/proc/loadavg: 6.87 6.55 5.70 6/186 30674
/proc/meminfo: memFree=28840064/32873844 swapFree=7628/7628
[pid=30616] ppid=30614 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/30616/stat : 30616 (run) S 30614 30616 29579 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 391142980 9367552 270 33554432000 4194304 5064252 140737436314000 140737436312912 251044456014 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/30616/statm: 2287 270 229 213 0 72 0
[pid=30617] ppid=30616 vsize=234076 CPUtime=1799.9 cores=1,3,5,7
/proc/30617/stat : 30617 (pb_cplex) S 30616 30616 29579 0 -1 4202496 110245 0 0 0 166421 13569 0 0 20 0 8 0 391142980 239693824 37226 33554432000 134512640 146736320 4287600480 4287596880 144580073 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0 0 0
/proc/30617/statm: 58519 37226 981 2985 0 55453 0
[pid=30617/tid=30641] ppid=30616 vsize=234076 CPUtime=192.88 cores=1,3,5,7
/proc/30617/task/30641/stat : 30641 (pb_cplex) R 30616 30616 29579 0 -1 4202560 2708 0 0 0 17810 1478 0 0 20 0 8 0 391150568 239693824 37226 33554432000 134512640 146736320 4287600480 4019396816 136008039 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30642] ppid=30616 vsize=234076 CPUtime=192.67 cores=1,3,5,7
/proc/30617/task/30642/stat : 30642 (pb_cplex) R 30616 30616 29579 0 -1 4202560 5279 0 0 0 17547 1720 0 0 20 0 8 0 391150568 239693824 37226 33554432000 134512640 146736320 4287600480 4029882784 136388288 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=30617/tid=30643] ppid=30616 vsize=234076 CPUtime=192.09 cores=1,3,5,7
/proc/30617/task/30643/stat : 30643 (pb_cplex) R 30616 30616 29579 0 -1 4202560 4106 0 0 0 17725 1484 0 0 20 0 8 0 391150568 239693824 37226 33554432000 134512640 146736320 4287600480 4040369176 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=30617/tid=30644] ppid=30616 vsize=234076 CPUtime=191.86 cores=1,3,5,7
/proc/30617/task/30644/stat : 30644 (pb_cplex) S 30616 30616 29579 0 -1 4202560 3170 0 0 0 17586 1600 0 0 20 0 8 0 391150568 239693824 37226 33554432000 134512640 146736320 4287600480 4120354448 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=30617/tid=30645] ppid=30616 vsize=234076 CPUtime=191.72 cores=1,3,5,7
/proc/30617/task/30645/stat : 30645 (pb_cplex) S 30616 30616 29579 0 -1 4202560 5770 0 0 0 17560 1612 0 0 20 0 8 0 391150568 239693824 37226 33554432000 134512640 146736320 4287600480 4070567568 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=30617/tid=30646] ppid=30616 vsize=234076 CPUtime=193.86 cores=1,3,5,7
/proc/30617/task/30646/stat : 30646 (pb_cplex) S 30616 30616 29579 0 -1 4202560 4063 0 0 0 17748 1638 0 0 20 0 8 0 391150568 239693824 37226 33554432000 134512640 146736320 4287600480 4060081808 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=30617/tid=30647] ppid=30616 vsize=234076 CPUtime=191.92 cores=1,3,5,7
/proc/30617/task/30647/stat : 30647 (pb_cplex) R 30616 30616 29579 0 -1 4202560 6460 0 0 0 17671 1521 0 0 20 0 8 0 391150568 239693824 37226 33554432000 134512640 146736320 4287600480 3985629252 136385175 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
Current children cumulated CPU time (s) 1799.9
Current children cumulated vsize (KiB) 243224

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 30616 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=0
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=999
# CPU time returned by wait4() is 0.000999
# while last known CPU time is 1799.9
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...


# WARNING:
# The CPU time of some children was not reported to their father
# (probably because of a missing or aborted wait()).
# This 'lost CPU time' is added to the watched process CPU time.
#  lost CPU time (s): 0.48999
#  lost CPU user time (s): 0.529999
#  lost CPU system time (s): -0.04

Real time (s): 515.617
CPU time (s): 1800.39
CPU user time (s): 1664.74
CPU system time (s): 135.65
CPU usage (%): 349.172
Max. virtual memory (cumulated for all children) (KiB): 243224

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0
system time used= 0.000999
maximum resident set size= 1080
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 355
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2
involuntary context switches= 1

runsolver used 1.46078 second user time and 3.36449 second system time

The end

Launcher Data 

Begin job on node143 at 2012-06-09 15:56:26
IDJOB=3741350
IDBENCH=89841
IDSOLVER=2325
FILE ID=node143/3741350-1339250186
RUNJOBID= node143-1339246834-29597
PBS_JOBID= 14644338
Free space on /tmp= 70080 MiB

SOLVER NAME= PB10: pb_cplex 2010-06-29
BENCH NAME= PB11/normalized-PB11/OPT-SMALLINT-LIN/heinz/normalized-p6b.opb
COMMAND LINE= run BENCHNAME
CONVERSION SCRIPT= PBconversionToLinear BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3741350-1339250186/watcher-3741350-1339250186 -o /tmp/evaluation-result-3741350-1339250186/solver-3741350-1339250186 -C 1800 -W 1900 -M 15500  run HOME/instance-3741350-1339250186.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f076140978d89a37e97e041a8b1e6b8f
RANDOM SEED=818103219

node143.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.787
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.787
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.787
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.787
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.787
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.787
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5298.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.787
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.787
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        29238680 kB
Buffers:         1931060 kB
Cached:           448592 kB
SwapCached:          196 kB
Active:          2394024 kB
Inactive:         190856 kB
Active(anon):     193184 kB
Inactive(anon):    11452 kB
Active(file):    2200840 kB
Inactive(file):   179404 kB
Unevictable:        7628 kB
Mlocked:            7628 kB
SwapTotal:      67108856 kB
SwapFree:       67108460 kB
Dirty:             16612 kB
Writeback:             0 kB
AnonPages:        213500 kB
Mapped:            11824 kB
Shmem:               160 kB
Slab:             904812 kB
SReclaimable:     346424 kB
SUnreclaim:       558388 kB
KernelStack:        1464 kB
PageTables:         4692 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     461088 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:    153600 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

CONVERSION COMMAND LINE= PBconversionToLinear HOME/instance-3741350-1339250186.opb

Free space on /tmp at the end= 70064 MiB
End job on node143 at 2012-06-09 16:05:02