Trace number 3725864

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
pwbo 2.02SAT (TO)47861 1800.08 900.319

General information on the benchmark

Namenormalized-PB06/OPT-SMALLINT/submitted-PB05/
manquinho/ttp/normalized-data8_3.opb
MD5SUMc3720ea5d84e92b781414b36064971fe
Bench CategoryOPT-SMALLINT (optimisation, small integers)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark43606
Best CPU time to get the best result obtained on this benchmark1800.05
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function 46478
Optimality of the best value was proved NO
Number of variables1344
Total number of constraints16096
Number of constraints which are clauses8208
Number of constraints which are cardinality constraints (but not clauses)496
Number of constraints which are nor clauses,nor cardinality constraints7392
Minimum length of a constraint2
Maximum length of a constraint28
Number of terms in the objective function 448
Biggest coefficient in the objective function 1380
Number of bits for the biggest coefficient in the objective function 11
Sum of the numbers in the objective function 279536
Number of bits of the sum of numbers in the objective function 19
Biggest number in a constraint 1380
Number of bits of the biggest number in a constraint 11
Biggest sum of numbers in a constraint 279536
Number of bits of the biggest sum of numbers19
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                     2.02 beta - May, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing opb file format. File HOME/instance-3725864-1338653236.opb.
0.00/0.00	c Instance file HOME/instance-3725864-1338653236.opb
0.00/0.00	c File size is 913860 bytes.
0.00/0.07	c Cost Offset: 0
0.00/0.07	c Highest Coefficient sum: 60
0.00/0.08	c ============================[ Problem Statistics ]=============================
0.00/0.08	c |                                                                             |
0.00/0.08	c |  Parsing time:          0.08         s                                      |
0.00/0.08	c |  Number Variables:      1344                                                |
0.00/0.08	c |  Number Clauses:        8488                                                |
0.00/0.08	c |  Number Cardinality:    496                                                 |
0.00/0.08	c |  Number PB Constraints: 7392                                                |
0.00/0.08	c |                                                                             |
0.00/0.08	c |  Problem Type:          Optimization                                        |
0.00/0.08	c |  Objective Function:    PB Constraint                                       |
0.00/0.08	c |                                                                             |
0.00/0.08	c ===============================================================================
1800.08/900.31	o 56335
1800.08/900.31	o 55407
1800.08/900.31	o 53674
1800.08/900.31	o 53054
1800.08/900.31	o 52983
1800.08/900.31	o 52938
1800.08/900.31	o 52566
1800.08/900.31	o 52099
1800.08/900.31	o 51979
1800.08/900.31	o 51791
1800.08/900.31	o 50270
1800.08/900.31	o 50203
1800.08/900.31	o 49126
1800.08/900.31	o 48883
1800.08/900.31	o 48685
1800.08/900.31	o 48372
1800.08/900.31	o 48284
1800.08/900.31	o 47861
1800.08/900.31	c ===============================================================================
1800.08/900.31	c Wall Clock Time: 900.31 s
1800.08/900.31	c Best Solution: 47861
1800.08/900.31	s SATISFIABLE
1800.08/900.31	v -x897 -x898 -x899 x900 x901 -x902 x903 -x904 -x905 -x906 -x907 -x908 x909 -x910 -x911 -x912 x913 -x914 -x915 -x916 -x917 x918 -x919 -x920 -x921 -x922 -x923 -x924 x925 -x926 -x927 -x928 -x929 -x930 -x931 -x932 x933 -x934 -x935 -x936 -x937 -x938 x939 -x940 -x941 -x942 -x943 -x944 -x945 -x946 -x947 x948 -x949 -x950 -x951 -x952 x953 -x954 -x955 -x956 -x957 -x958 -x959 x960 x961 x962 -x963 x964 -x965 -x966 -x967 -x968 -x969 -x970 -x971 -x972 x973 -x974 -x975 -x976 -x977 -x978 x979 -x980 -x981 x982 -x983 -x984 -x985 -x986 -x987 -x988 x989 -x990 -x991 -x992 -x993 -x994 -x995 x996 -x997 -x998 -x999 -x1000 -x1001 -x1002 -x1003 -x1004 -x1005 x1006 -x1007 -x1008 -x1009 -x1010 -x1011 -x1012 -x1013 -x1014 x1015 -x1016 -x1017 -x1018 -x1019 -x1020 x1021 -x1022 x1023 x1024 x1025 -x1026 -x1027 -x1028 -x1029 -x1030 -x1031 x1032 -x1033 -x1034 -x1035 -x1036 -x1037 -x1038 x1039 -x1040 -x1041 -x1042 -x1043 -x1044 -x1045 x1046 -x1047 -x1048 -x1049 -x1050 -x1051 -x1052 -x1053 -x1054 x1055 -x1056 -x1057 -x1058 -x1059 -x1060 -x1061 -x1062 x1063 -x1064 -x1065 -x1066 x1067 -x1068 -x1069 -x1070 -x1071 x1072 -x1073 -x1074 -x1075 -x1076 -x1077 -x1078 -x1079 -x1080 -x1081 x1082 -x1083 -x1084 -x1085 -x1086 x1087 x1088 -x1089 -x1090 x1091 -x1092 -x1093 -x1094 -x1095 -x1096 -x1097 -x1098 x1099 -x1100 -x1101 -x1102 x1103 -x1104 -x1105 -x1106 -x1107 -x1108 -x1109 -x1110 -x1111 x1112 -x1113 -x1114 -x1115 -x1116 x1117 -x1118 -x1119 -x1120 -x1121 -x1122 -x1123 -x1124 -x1125 -x1126 x1127 -x1128 -x1129 -x1130 x1131 -x1132 -x1133 -x1134 x1135 -x1136 -x1137 -x1138 -x1139 -x1140 -x1141 -x1142 -x1143 -x1144 -x1145 x1146 -x1147 -x1148 -x1149 -x1150 x1151 x1152 -x1153 x1154 -x1155 -x1156 -x1157 -x1158 -x1159 x1160 -x1161 -x1162 -x1163 x1164 -x1165 -x1166 -x1167 -x1168 -x1169 -x1170 -x1171 -x1172 -x1173 x1174 -x1175 -x1176 -x1177 -x1178 -x1179 x1180 -x1181 -x1182 -x1183 -x1184 -x1185 -x1186 -x1187 x1188 -x1189 -x1190 -x1191 x1192 -x1193 -x1194 -x1195 -x1196 -x1197 -x1198 -x1199 -x1200 -x1201 x1202 -x1203 -x1204 -x1205 -x1206 -x1207 -x1208 x1209 -x1210 -x1211 -x1212 -x1213 x1214 x1215 -x1216 x1217 x1218 -x1219 -x1220 -x1221 -x1222 -x1223 x1224 -x1225 x1226 -x1227 -x1228 -x1229 -x1230 -x1231 -x1232 -x1233 -x1234 -x1235 x1236 -x1237 -x1238 -x1239 -x1240 x1241 -x1242 -x1243 -x1244 -x1245 -x1246 -x1247 -x1248 -x1249 -x1250 -x1251 x1252 -x1253 -x1254 -x1255 -x1256 -x1257 -x1258 -x1259 x1260 -x1261 -x1262 -x1263 -x1264 -x1265 x1266 -x1267 -x1268 -x1269 -x1270 x1271 -x1272 -x1273 -x1274 x1275 x1276 -x1277 -x1278 -x1279 x1280 -x1281 -x1282 -x1283 -x1284 -x1285 -x1286 -x1287 x1288 -x1289 -x1290 -x1291 -x1292 -x1293 -x1294 x1295 -x1296 x1297 -x1298 -x1299 -x1300 -x1301 -x1302 -x1303 -x1304 -x1305 -x1306 -x1307 -x1308 x1309 -x1310 x1311 -x1312 -x1313 -x1314 -x1315 -x1316 -x1317 -x1318 -x1319 -x1320 -x1321 -x1322 x1323 x1324 -x1325 -x1326 -x1327 -x1328 -x1329 -x1330 -x1331 -x1332 -x1333 -x1334 -x1335 x1336 -x1337 -x1338 -x1339 -x1340 x1341 x1342 -x1343 x1344 -x2 -x66 -x130 -x194 -x258 -x322 -x386 x450 -x514 -x578 -x642 -x706 -x770 -x834 -x3 x67 -x131 -x195 -x259 -x323 -x387 -x451 -x515 -x579 -x643 -x707 -x771 -x835 -x4 -x68 x132 -x196 -x260 -x324 -x388 -x452 -x516 -x580 -x644 -x708 -x772 -x836 -x5 -x69 -x133 -x197 -x261 -x325 -x389 -x453 -x517 -x581 -x645 -x709 x773 -x837 -x6 -x70 -x134 -x198 -x262 -x326 x390 -x454 -x518 -x582 -x646 -x710 -x774 -x838 -x7 -x71 -x135 -x199 -x263 -x327 -x391 -x455 x519 -x583 -x647 -x711 -x775 -x839 x8 -x72 -x136 -x200 -x264 -x328 -x392 -x456 -x520 -x584 -x648 -x712 -x776 -x840 -x9 -x73 -x137 -x201 x265 -x329 -x393 -x457 -x521 -x585 -x649 -x713 -x777 -x841 x11 -x75 -x139 -x203 -x267 -x331 -x395 -x459 -x523 -x587 -x651 -x715 -x779 -x843 -x12 -x76 -x140 -x204 -x268 -x332 -x396 -x460 -x524 -x588 -x652 -x716 x780 -x844 -x13 -x77 x141 -x205 -x269 -x333 -x397 -x461 -x525 -x589 -x653 -x717 -x781 -x845 -x14 -x78 -x142 -x206 -x270 -x334 -x398 -x462 x526 -x590 -x654 -x718 -x782 -x846 -x15 -x79 -x143 -x207 -x271 -x335 -x399 -x463 -x527 -x591 -x655 -x719 -x783 x847 -x16 x80 -x144 -x208 -x272 -x336 -x400 -x464 -x528 -x592 -x656 -x720 -x784 -x848 -x17 -x81 -x145 x209 -x273 -x337 -x401 -x465 -x529 -x593 -x657 -x721 -x785 -x849 -x18 -x82 -x146 -x210 -x274 -x338 -x402 -x466 -x530 -x594 -x658 x722 -x786 -x850 -x20 -x84 -x148 -x212 -x276 -x340 -x404 -x468 -x532 x596 -x660 -x724 -x788 -x852 -x21 -x85 -x149 -x213 -x277 -x341 -x405 -x469 -x533 -x597 -x661 -x725 -x789 x853 -x22 -x86 -x150 -x214 -x278 x342 -x406 -x470 -x534 -x598 -x662 -x726 -x790 -x854 -x23 -x87 -x151 -x215 -x279 -x343 -x407 -x471 -x535 -x599 -x663 -x727 x791 -x855 -x24 -x88 -x152 -x216 x280 -x344 -x408 -x472 -x536 -x600 -x664 -x728 -x792 -x856 -x25 -x89 -x153 -x217 -x281 -x345 -x409 -x473 -x537 -x601 -x665 x729 -x793 -x857 -x26 -x90 -x154 -x218 -x282 -x346 -x410 -x474 -x538 -x602 x666 -x730 -x794 -x858 -x27 -x91 -x155 -x219 -x283 -x347 x411 -x475 -x539 -x603 -x667 -x731 -x795 -x859 -x29 -x93 -x157 -x221 -x285 -x349 -x413 x477 -x541 -x605 -x669 -x733 -x797 -x861 -x30 -x94 -x158 -x222 x286 -x350 -x414 -x478 -x542 -x606 -x670 -x734 -x798 -x862 -x31 -x95 -x159 x223 -x287 -x351 -x415 -x479 -x543 -x607 -x671 -x735 -x799 -x863 -x32 -x96 -x160 -x224 -x288 -x352 -x416 -x480 x544 -x608 -x672 -x736 -x800 -x864 -x33 -x97 -x161 -x225 -x289 -x353 -x417 -x481 -x545 x609 -x673 -x737 -x801 -x865 -x34 -x98 -x162 -x226 -x290 x354 -x418 -x482 -x546 -x610 -x674 -x738 -x802 -x866 -x35 -x99 -x163 -x227 -x291 -x355 -x419 -x483 x547 -x611 -x675 -x739 -x803 -x867 x36 -x100 -x164 -x228 -x292 -x356 -x420 -x484 -x548 -x612 -x676 -x740 -x804 -x868 -x38 x102 -x166 -x230 -x294 -x358 -x422 -x486 -x550 -x614 -x678 -x742 -x806 -x870 -x39 -x103 -x167 -x231 x295 -x359 -x423 -x487 -x551 -x615 -x679 -x743 -x807 -x871 -x40 -x104 -x168 -x232 -x296 -x360 -x424 -x488 -x552 -x616 x680 -x744 -x808 -x872 -x41 -x105 -x169 -x233 -x297 -x361 -x425 -x489 -x553 -x617 x681 -x745 -x809 -x873 -x42 -x106 -x170 x234 -x298 -x362 -x426 -x490 -x554 -x618 -x682 -x746 -x810 -x874 -x43 -x107 x171 -x235 -x299 -x363 -x427 -x491 -x555 -x619 -x683 -x747 -x811 -x875 -x44 -x108 -x172 -x236 -x300 -x364 -x428 -x492 -x556 -x620 -x684 -x748 -x812 x876 -x45 -x109 -x173 -x237 -x301 -x365 -x429 -x493 -x557 -x621 -x685 x749 -x813 -x877 -x47 -x111 -x175 -x239 -x303 -x367 -x431 x495 -x559 -x623 -x687 -x751 -x815 -x879 -x48 -x112 -x176 -x240 -x304 -x368 -x432 -x496 -x560 x624 -x688 -x752 -x816 -x880 -x49 -x113 -x177 -x241 -x305 x369 -x433 -x497 -x561 -x625 -x689 -x753 -x817 -x881 -x50 -x114 -x178 -x242 -x306 -x370 -x434 -x498 -x562 x626 -x690 -x754 -x818 -x882 -x51 -x115 -x179 -x243 -x307 -x371 -x435 -x499 -x563 -x627 x691 -x755 -x819 -x883 -x52 x116 -x180 -x244 -x308 -x372 -x436 -x500 -x564 -x628 -x692 -x756 -x820 -x884 -x53 -x117 -x181 -x245 -x309 -x373 x437 -x501 -x565 -x629 -x693 -x757 -x821 -x885 x54 -x118 -x182 -x246 -x310 -x374 -x438 -x502 -x566 -x630 -x694 -x758 -x822 -x886 -x56 -x120 x184 -x248 -x312 -x376 -x440 -x504 -x568 -x632 -x696 -x760 -x824 -x888 -x57 -x121 -x185 -x249 -x313 -x377 -x441 -x505 -x569 -x633 -x697 -x761 -x825 x889 -x58 -x122 -x186 -x250 -x314 -x378 x442 -x506 -x570 -x634 -x698 -x762 -x826 -x890 -x59 -x123 -x187 -x251 -x315 -x379 -x443 x507 -x571 -x635 -x699 -x763 -x827 -x891 -x60 -x124 -x188 -x252 -x316 x380 -x444 -x508 -x572 -x636 -x700 -x764 -x828 -x892 -x61 -x125 -x189 x253 -x317 -x381 -x445 -x509 -x573 -x637 -x701 -x765 -x829 -x893 -x62 -x126 -x190 -x254 -x318 -x382 -x446 -x510 -x574 -x638 -x702 -x766 x830 -x894 -x63 -x127 -x191 -x255 -x319 -x383 -x447 -x511 -x575 -x639 -x703 x767 -x831 -x895 -x10 -x74 -x138 x202 -x266 x330 x394 x458 -x522 x586 x650 x714 -x778 -x842 x19 x83 x147 -x211 -x275 -x339 x403 x467 x531 -x595 x659 -x723 -x787 -x851 x28 x92 x156 -x220 -x284 x348 -x412 -x476 -x540 x604 -x668 -x732 x796 x860 -x37 -x101 x165 x229 -x293 -x357 x421 x485 -x549 -x613 -x677 x741 x805 x869 x46 x110 -x174 -x238 x302 x366 x430 -x494 x558 -x622 -x686 -x750 x814 -x878 -x55 -x119 -x183 x247 x311 -x375 -x439 x503 x567 -x631 -x695 x759 x823 x887 x64 x128 x192 -x256 x320 -x384 -x448 -x512 x576 x640 x704 -x768 -x832 -x896 -x1 -x65 -x129 x193 x257 x321 -x385 -x449 -x513 x577 x641 x705 -x769 x833

Verifier Data

OK	47861

Watcher Data

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3725864-1338653236/watcher-3725864-1338653236 -o /tmp/evaluation-result-3725864-1338653236/solver-3725864-1338653236 -C 1800 -W 1900 -M 15500 pwbo2.02 -file-format=opb -verbosity=0 HOME/instance-3725864-1338653236.opb 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.00 3.98 3.58 4/178 29881
/proc/meminfo: memFree=27423784/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=5692 CPUtime=0 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) R 29879 29881 28266 0 -1 4202496 534 0 0 0 0 0 0 0 20 0 1 0 331448433 5828608 486 33554432000 4194304 5964478 140735092353120 140735379733016 5136490 0 0 4096 16387 0 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 1423 487 169 433 0 763 0

[startup+0.0843281 s]
/proc/loadavg: 4.00 3.98 3.58 4/178 29881
/proc/meminfo: memFree=27423784/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=173828 CPUtime=0.08 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 4186 0 0 0 8 0 0 0 20 0 3 0 331448433 177999872 3911 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 43457 3911 187 433 0 43021 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 173828

[startup+0.100215 s]
/proc/loadavg: 4.00 3.98 3.58 4/178 29881
/proc/meminfo: memFree=27423784/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=174236 CPUtime=0.1 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 4709 0 0 0 10 0 0 0 20 0 3 0 331448433 178417664 4434 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 43559 4434 191 433 0 43123 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 174236

[startup+0.300207 s]
/proc/loadavg: 4.00 3.98 3.58 4/178 29881
/proc/meminfo: memFree=27423784/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=173948 CPUtime=0.5 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 8854 0 0 0 49 1 0 0 20 0 3 0 331448433 178122752 6983 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 43487 6983 199 433 0 43051 0
Current children cumulated CPU time (s) 0.5
Current children cumulated vsize (KiB) 173948

[startup+0.700224 s]
/proc/loadavg: 4.00 3.98 3.58 4/178 29881
/proc/meminfo: memFree=27423784/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=173948 CPUtime=1.3 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 13351 0 0 0 128 2 0 0 20 0 3 0 331448433 178122752 10561 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 43487 10561 199 433 0 43051 0
Current children cumulated CPU time (s) 1.3
Current children cumulated vsize (KiB) 173948

[startup+1.50038 s]
/proc/loadavg: 4.00 3.98 3.58 5/181 29884
/proc/meminfo: memFree=27365240/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=173948 CPUtime=2.89 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 19787 0 0 0 282 7 0 0 20 0 3 0 331448433 178122752 16997 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 43487 16997 199 433 0 43051 0
[pid=29881/tid=29883] ppid=29879 vsize=173948 CPUtime=1.39 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 12292 0 0 0 133 6 0 0 20 0 3 0 331448441 178122752 16997 33554432000 4194304 5964478 140735092353120 139706815068504 4445054 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=173948 CPUtime=1.4 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 3956 0 0 0 140 0 0 0 20 0 3 0 331448441 178122752 16997 33554432000 4194304 5964478 140735092353120 139706804578664 5136420 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 2.89
Current children cumulated vsize (KiB) 173948

[startup+3.10037 s]
/proc/loadavg: 4.00 3.98 3.58 5/181 29884
/proc/meminfo: memFree=27328520/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=239484 CPUtime=6.09 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 29159 0 0 0 596 13 0 0 20 0 3 0 331448433 245231616 25372 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 59871 25372 199 433 0 59435 0
[pid=29881/tid=29883] ppid=29879 vsize=239484 CPUtime=2.99 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 20563 0 0 0 287 12 0 0 20 0 3 0 331448441 245231616 25372 33554432000 4194304 5964478 140735092353120 139706815068696 5139118 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=239484 CPUtime=2.99 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 5057 0 0 0 299 0 0 0 20 0 3 0 331448441 245231616 25372 33554432000 4194304 5964478 140735092353120 139706804578968 4403708 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 6.09
Current children cumulated vsize (KiB) 239484

[startup+6.30043 s]
/proc/loadavg: 4.00 3.98 3.58 5/181 29884
/proc/meminfo: memFree=27262800/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=239484 CPUtime=12.49 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 41645 0 0 0 1227 22 0 0 20 0 3 0 331448433 245231616 37459 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 59871 37459 199 433 0 59435 0
[pid=29881/tid=29883] ppid=29879 vsize=239484 CPUtime=6.18 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 31869 0 0 0 598 20 0 0 20 0 3 0 331448441 245231616 37459 33554432000 4194304 5964478 140735092353120 139706815068504 5142509 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=239484 CPUtime=6.18 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 6237 0 0 0 617 1 0 0 20 0 3 0 331448441 245231616 37459 33554432000 4194304 5964478 140735092353120 139706804578616 4400115 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 12.49
Current children cumulated vsize (KiB) 239484

[startup+12.7004 s]
/proc/loadavg: 4.00 3.98 3.59 5/181 29884
/proc/meminfo: memFree=27167732/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=370556 CPUtime=25.29 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 61166 0 0 0 2501 28 0 0 20 0 3 0 331448433 379449344 56443 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 92639 56444 199 433 0 92203 0
[pid=29881/tid=29883] ppid=29879 vsize=370556 CPUtime=12.56 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 49926 0 0 0 1231 25 0 0 20 0 3 0 331448441 379449344 56444 33554432000 4194304 5964478 140735092353120 139706815068456 5139112 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=370556 CPUtime=12.57 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 7702 0 0 0 1255 2 0 0 20 0 3 0 331448441 379449344 56444 33554432000 4194304 5964478 140735092353120 139706804578664 5136420 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 25.29
Current children cumulated vsize (KiB) 370556

[startup+25.5004 s]
/proc/loadavg: 4.00 3.98 3.59 5/181 29884
/proc/meminfo: memFree=27044352/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=436620 CPUtime=50.88 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 94820 0 0 0 5052 36 0 0 20 0 3 0 331448433 447098880 82047 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 109155 82047 199 433 0 108719 0
[pid=29881/tid=29883] ppid=29879 vsize=436620 CPUtime=25.33 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 80618 0 0 0 2502 31 0 0 20 0 3 0 331448441 447098880 82047 33554432000 4194304 5964478 140735092353120 139706815068504 5179577 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=436620 CPUtime=25.33 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 10663 0 0 0 2529 4 0 0 20 0 3 0 331448441 447098880 82047 33554432000 4194304 5964478 140735092353120 139706804578968 4400010 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 50.88
Current children cumulated vsize (KiB) 436620

[startup+51.1004 s]
/proc/loadavg: 4.00 3.98 3.60 5/181 29884
/proc/meminfo: memFree=26975736/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=437412 CPUtime=102.07 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 104934 0 0 0 10168 39 0 0 20 0 3 0 331448433 447909888 90962 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 109353 90962 199 433 0 108917 0
[pid=29881/tid=29883] ppid=29879 vsize=437412 CPUtime=50.86 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 90289 0 0 0 5052 34 0 0 20 0 3 0 331448441 447909888 90962 33554432000 4194304 5964478 140735092353120 139706815068064 4399920 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=437412 CPUtime=50.87 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 11106 0 0 0 5083 4 0 0 20 0 3 0 331448441 447909888 90962 33554432000 4194304 5964478 140735092353120 139706804578376 4400001 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 102.07
Current children cumulated vsize (KiB) 437412

[startup+102.307 s]
/proc/loadavg: 4.00 3.98 3.62 5/181 29885
/proc/meminfo: memFree=26869092/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=504164 CPUtime=204.44 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 119179 0 0 0 20397 47 0 0 20 0 3 0 331448433 516263936 105052 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 126041 105052 199 433 0 125605 0
[pid=29881/tid=29883] ppid=29879 vsize=504164 CPUtime=101.92 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 95222 0 0 0 10154 38 0 0 20 0 3 0 331448441 516263936 105052 33554432000 4194304 5964478 140735092353120 139706815068152 4379120 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=504164 CPUtime=101.94 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 20418 0 0 0 10184 10 0 0 20 0 3 0 331448441 516263936 105052 33554432000 4194304 5964478 140735092353120 139706804578968 4222416 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 204.44
Current children cumulated vsize (KiB) 504164

[startup+162.3 s]
/proc/loadavg: 4.00 3.98 3.64 5/181 29885
/proc/meminfo: memFree=26832912/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=505376 CPUtime=324.38 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 121754 0 0 0 32390 48 0 0 20 0 3 0 331448433 517505024 107098 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 126344 107098 199 433 0 125908 0
[pid=29881/tid=29883] ppid=29879 vsize=505376 CPUtime=161.72 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 97463 0 0 0 16132 40 0 0 20 0 3 0 331448441 517505024 107098 33554432000 4194304 5964478 140735092353120 139706815068064 4400056 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=505376 CPUtime=161.78 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 20752 0 0 0 16167 11 0 0 20 0 3 0 331448441 517505024 107098 33554432000 4194304 5964478 140735092353120 139706804578720 4399923 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 324.38
Current children cumulated vsize (KiB) 505376

[startup+222.3 s]
/proc/loadavg: 4.00 3.98 3.66 5/182 29888
/proc/meminfo: memFree=26761880/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=506048 CPUtime=444.35 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 131336 0 0 0 44377 58 0 0 20 0 3 0 331448433 518193152 116673 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 126512 116673 199 433 0 126076 0
[pid=29881/tid=29883] ppid=29879 vsize=506048 CPUtime=221.56 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 106888 0 0 0 22105 51 0 0 20 0 3 0 331448441 518193152 116673 33554432000 4194304 5964478 140735092353120 139706815068200 4400087 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=506048 CPUtime=221.62 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 20909 0 0 0 22150 12 0 0 20 0 3 0 331448441 518193152 116673 33554432000 4194304 5964478 140735092353120 139706804578376 4403376 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 444.35
Current children cumulated vsize (KiB) 506048

[startup+282.3 s]
/proc/loadavg: 4.00 3.98 3.68 5/181 29888
/proc/meminfo: memFree=26701156/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=638220 CPUtime=564.31 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 140393 0 0 0 56368 63 0 0 20 0 3 0 331448433 653537280 125638 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 159555 125638 199 433 0 159119 0
[pid=29881/tid=29883] ppid=29879 vsize=638220 CPUtime=281.4 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 110454 0 0 0 28087 53 0 0 20 0 3 0 331448441 653537280 125638 33554432000 4194304 5964478 140735092353120 139706815068696 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=638220 CPUtime=281.44 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 26400 0 0 0 28127 17 0 0 20 0 3 0 331448441 653537280 125638 33554432000 4194304 5964478 140735092353120 139706804578376 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 564.31
Current children cumulated vsize (KiB) 638220

[startup+342.3 s]
/proc/loadavg: 4.12 4.01 3.71 5/181 29888
/proc/meminfo: memFree=26641872/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=639032 CPUtime=684.28 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 144411 0 0 0 68361 67 0 0 20 0 3 0 331448433 654368768 130629 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 159758 130629 199 433 0 159322 0

################
# More data... #
################

[pid=29881/tid=29884] ppid=29879 vsize=639032 CPUtime=341.28 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 30345 0 0 0 34106 22 0 0 20 0 3 0 331448441 654368768 130629 33554432000 4194304 5964478 140735092353120 139706804578376 4400010 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 684.28
Current children cumulated vsize (KiB) 639032

[startup+402.3 s]
/proc/loadavg: 4.04 4.00 3.72 5/181 29888
/proc/meminfo: memFree=26597276/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=639884 CPUtime=804.26 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 148708 0 0 0 80353 73 0 0 20 0 3 0 331448433 655241216 134906 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 159971 134906 199 433 0 159535 0
[pid=29881/tid=29883] ppid=29879 vsize=639884 CPUtime=401.09 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 110534 0 0 0 40054 55 0 0 20 0 3 0 331448441 655241216 134906 33554432000 4194304 5964478 140735092353120 139706815068064 4400087 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=639884 CPUtime=401.13 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 34635 0 0 0 40084 29 0 0 20 0 3 0 331448441 655241216 134906 33554432000 4194304 5964478 140735092353120 139706804578376 4399920 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 804.26
Current children cumulated vsize (KiB) 639884

[startup+462.3 s]
/proc/loadavg: 4.01 4.00 3.74 5/181 29889
/proc/meminfo: memFree=26578360/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=640744 CPUtime=924.25 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 150948 0 0 0 92349 76 0 0 20 0 3 0 331448433 656121856 137146 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 160186 137146 199 433 0 159750 0
[pid=29881/tid=29883] ppid=29879 vsize=640744 CPUtime=460.93 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 110610 0 0 0 46038 55 0 0 20 0 3 0 331448441 656121856 137146 33554432000 4194304 5964478 140735092353120 139706815068696 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=640744 CPUtime=460.98 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 36799 0 0 0 46065 33 0 0 20 0 3 0 331448441 656121856 137146 33554432000 4194304 5964478 140735092353120 139706804578616 4392968 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 924.25
Current children cumulated vsize (KiB) 640744

[startup+522.3 s]
/proc/loadavg: 4.00 4.00 3.75 5/181 29889
/proc/meminfo: memFree=26540568/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=641440 CPUtime=1044.22 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 154009 0 0 0 104341 81 0 0 20 0 3 0 331448433 656834560 140311 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 160360 140311 199 433 0 159924 0
[pid=29881/tid=29883] ppid=29879 vsize=641440 CPUtime=520.78 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 110622 0 0 0 52022 56 0 0 20 0 3 0 331448441 656834560 140311 33554432000 4194304 5964478 140735092353120 139706815068696 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=641440 CPUtime=520.81 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 39848 0 0 0 52043 38 0 0 20 0 3 0 331448441 656834560 140311 33554432000 4194304 5964478 140735092353120 139706804578376 4399920 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1044.22
Current children cumulated vsize (KiB) 641440

[startup+582.3 s]
/proc/loadavg: 4.00 4.00 3.76 5/181 29889
/proc/meminfo: memFree=26531240/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=707556 CPUtime=1164.17 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 156217 0 0 0 116333 84 0 0 20 0 3 0 331448433 724537344 142277 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 176889 142277 199 433 0 176453 0
[pid=29881/tid=29883] ppid=29879 vsize=707556 CPUtime=580.6 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 110647 0 0 0 58003 57 0 0 20 0 3 0 331448441 724537344 142277 33554432000 4194304 5964478 140735092353120 139706815068064 4221120 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=707556 CPUtime=580.64 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 42031 0 0 0 58023 41 0 0 20 0 3 0 331448441 724537344 142277 33554432000 4194304 5964478 140735092353120 139706804578720 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1164.17
Current children cumulated vsize (KiB) 707556

[startup+642.3 s]
/proc/loadavg: 4.00 4.00 3.77 5/181 29889
/proc/meminfo: memFree=26510324/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=707960 CPUtime=1284.16 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 156314 0 0 0 128332 84 0 0 20 0 3 0 331448433 724951040 142433 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 176990 142433 199 433 0 176554 0
[pid=29881/tid=29883] ppid=29879 vsize=707960 CPUtime=640.44 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 110658 0 0 0 63986 58 0 0 20 0 3 0 331448441 724951040 142433 33554432000 4194304 5964478 140735092353120 139706815068064 4403590 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=707960 CPUtime=640.49 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 42117 0 0 0 64007 42 0 0 20 0 3 0 331448441 724951040 142433 33554432000 4194304 5964478 140735092353120 139706804578376 4400067 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1284.16
Current children cumulated vsize (KiB) 707960

[startup+702.3 s]
/proc/loadavg: 4.00 4.00 3.78 5/181 29889
/proc/meminfo: memFree=26490112/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=708380 CPUtime=1404.13 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 156419 0 0 0 140329 84 0 0 20 0 3 0 331448433 725381120 142538 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 177095 142538 199 433 0 176659 0
[pid=29881/tid=29883] ppid=29879 vsize=708380 CPUtime=700.28 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 110662 0 0 0 69969 59 0 0 20 0 3 0 331448441 725381120 142538 33554432000 4194304 5964478 140735092353120 139706815068064 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=708380 CPUtime=700.33 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 42218 0 0 0 69990 43 0 0 20 0 3 0 331448441 725381120 142538 33554432000 4194304 5964478 140735092353120 139706804578376 4400089 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1404.13
Current children cumulated vsize (KiB) 708380

[startup+762.307 s]
/proc/loadavg: 4.00 4.00 3.80 5/181 29890
/proc/meminfo: memFree=26451300/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=708976 CPUtime=1524.12 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 161600 0 0 0 152326 86 0 0 20 0 3 0 331448433 725991424 147717 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 177244 147717 199 433 0 176808 0
[pid=29881/tid=29883] ppid=29879 vsize=708976 CPUtime=760.14 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 115713 0 0 0 75952 62 0 0 20 0 3 0 331448441 725991424 147717 33554432000 4194304 5964478 140735092353120 139706815068504 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=708976 CPUtime=760.18 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 42348 0 0 0 75974 44 0 0 20 0 3 0 331448441 725991424 147717 33554432000 4194304 5964478 140735092353120 139706804578376 4400087 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1524.12
Current children cumulated vsize (KiB) 708976

[startup+822.3 s]
/proc/loadavg: 4.00 4.00 3.81 5/182 29893
/proc/meminfo: memFree=26431948/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=709396 CPUtime=1644.09 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 161715 0 0 0 164323 86 0 0 20 0 3 0 331448433 726421504 147832 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 177349 147832 199 433 0 176913 0
[pid=29881/tid=29883] ppid=29879 vsize=709396 CPUtime=819.98 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 115714 0 0 0 81935 63 0 0 20 0 3 0 331448441 726421504 147832 33554432000 4194304 5964478 140735092353120 139706815068504 4400044 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=709396 CPUtime=820.02 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 42462 0 0 0 81957 45 0 0 20 0 3 0 331448441 726421504 147832 33554432000 4194304 5964478 140735092353120 139706804578376 4399920 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1644.09
Current children cumulated vsize (KiB) 709396

[startup+882.3 s]
/proc/loadavg: 4.00 4.00 3.82 5/181 29893
/proc/meminfo: memFree=26410952/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=710440 CPUtime=1764.07 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 161967 0 0 0 176321 86 0 0 20 0 3 0 331448433 727490560 148084 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 177610 148084 199 433 0 177174 0
[pid=29881/tid=29883] ppid=29879 vsize=710440 CPUtime=879.82 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 115730 0 0 0 87918 64 0 0 20 0 3 0 331448441 727490560 148084 33554432000 4194304 5964478 140735092353120 139706815068504 4400091 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=710440 CPUtime=879.87 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 42698 0 0 0 87940 47 0 0 20 0 3 0 331448441 727490560 148084 33554432000 4194304 5964478 140735092353120 139706804578376 4222330 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1764.07
Current children cumulated vsize (KiB) 710440



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.3 s]
/proc/loadavg: 4.00 4.00 3.82 3/177 29894
/proc/meminfo: memFree=26839268/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=710440 CPUtime=1800.08 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 163277 0 0 0 179920 88 0 0 20 0 3 0 331448433 727490560 149394 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 177610 149394 199 433 0 177174 0
[pid=29881/tid=29883] ppid=29879 vsize=710440 CPUtime=897.78 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 115730 0 0 0 89714 64 0 0 20 0 3 0 331448441 727490560 149394 33554432000 4194304 5964478 140735092353120 139706815068504 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=710440 CPUtime=897.83 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 44008 0 0 0 89734 49 0 0 20 0 3 0 331448441 727490560 149394 33554432000 4194304 5964478 140735092353120 139706804578616 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1800.08
Current children cumulated vsize (KiB) 710440

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+900.3 s]
/proc/loadavg: 4.00 4.00 3.82 3/177 29894
/proc/meminfo: memFree=26839268/32873844 swapFree=6608/6608
[pid=29881] ppid=29879 vsize=710440 CPUtime=1800.08 cores=1,3,5,7
/proc/29881/stat : 29881 (pwbo2.02) S 29879 29881 28266 0 -1 4202496 163277 0 0 0 179920 88 0 0 20 0 3 0 331448433 727490560 149394 33554432000 4194304 5964478 140735092353120 140735092352640 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/29881/statm: 177610 149394 199 433 0 177174 0
[pid=29881/tid=29883] ppid=29879 vsize=710440 CPUtime=897.78 cores=1,3,5,7
/proc/29881/task/29883/stat : 29883 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 115730 0 0 0 89714 64 0 0 20 0 3 0 331448441 727490560 149394 33554432000 4194304 5964478 140735092353120 139706815068504 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=29881/tid=29884] ppid=29879 vsize=710440 CPUtime=897.83 cores=1,3,5,7
/proc/29881/task/29884/stat : 29884 (pwbo2.02) R 29879 29881 28266 0 -1 4202560 44008 0 0 0 89734 49 0 0 20 0 3 0 331448441 727490560 149394 33554432000 4194304 5964478 140735092353120 139706804578616 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1800.08
Current children cumulated vsize (KiB) 710440

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 29881 and gives
#  childrusage.ru_utime.tv_sec=1794
#  childrusage.ru_utime.tv_usec=561185
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=164822
# CPU time returned by wait4() is 1795.73
# while last known CPU time is 1800.08
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 900.319
CPU time (s): 1800.08
CPU user time (s): 1799.2
CPU system time (s): 0.88
CPU usage (%): 199.938
Max. virtual memory (cumulated for all children) (KiB): 710440

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1794.56
system time used= 1.16482
maximum resident set size= 597588
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 163280
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 348
involuntary context switches= 2331

runsolver used 1.9607 second user time and 5.46417 second system time

The end

Launcher Data

Begin job on node140 at 2012-06-02 18:07:16
IDJOB=3725864
IDBENCH=2008
IDSOLVER=2309
FILE ID=node140/3725864-1338653236
RUNJOBID= node140-1338651425-29650
PBS_JOBID= 14636440
Free space on /tmp= 70620 MiB

SOLVER NAME= pwbo 2.02
BENCH NAME= PB06//final/normalized-PB06/OPT-SMALLINT/submitted-PB05/manquinho/ttp/normalized-data8_3.opb
COMMAND LINE= pwbo2.02 -file-format=opb -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3725864-1338653236/watcher-3725864-1338653236 -o /tmp/evaluation-result-3725864-1338653236/solver-3725864-1338653236 -C 1800 -W 1900 -M 15500  pwbo2.02 -file-format=opb -verbosity=0 HOME/instance-3725864-1338653236.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= c3720ea5d84e92b781414b36064971fe
RANDOM SEED=1373807438

node140.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.88
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        27424692 kB
Buffers:          389940 kB
Cached:          4146792 kB
SwapCached:         1664 kB
Active:          1237540 kB
Inactive:        3324612 kB
Active(anon):      17044 kB
Inactive(anon):     9416 kB
Active(file):    1220496 kB
Inactive(file):  3315196 kB
Unevictable:        6608 kB
Mlocked:            6608 kB
SwapTotal:      67108856 kB
SwapFree:       67105156 kB
Dirty:             11920 kB
Writeback:             0 kB
AnonPages:         29712 kB
Mapped:            10920 kB
Shmem:                 8 kB
Slab:             747304 kB
SReclaimable:     189268 kB
SUnreclaim:       558036 kB
KernelStack:        1424 kB
PageTables:         3576 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     158996 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70616 MiB
End job on node140 at 2012-06-02 18:22:18