(3) 18:34:28 Main [DGramLis] >> Listening for UDP datagrams: Port 35293
(3) 18:34:28 Main [App] >> Version: 2012.10-r10
(3) 18:34:28 Main [App] >> LocalNode: compute-3-5.local
(3) 18:34:28 Main [App] >> Usable NICs: ib0(RDMA) ib0(TCP) eth1(TCP) eth0(TCP) 
(2) 18:34:28 DGramLis [Heartbeat incoming] >> New node: fhgfs-mgmtd fs0 [ID: 1]; Source: 10.2.7.50
(3) 18:34:28 Main [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.50:8008
(3) 18:34:28 Main [NodeConn (acquire stream)] >> Connected: 10.2.7.50:8008
(3) 18:34:28 Main [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.50:8005
(3) 18:34:28 Main [NodeConn (acquire stream)] >> Connected: 10.2.7.50:8005
(3) 18:34:28 Main [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.15:8003
(3) 18:34:28 Main [NodeConn (acquire stream)] >> Connected: 10.2.7.15:8003
(3) 18:34:28 Main [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.16:8003
(3) 18:34:28 Main [NodeConn (acquire stream)] >> Connected: 10.2.7.16:8003
(3) 18:34:44 Worker3 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.50:8005
(3) 18:34:44 Worker11 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.50:8005
(3) 18:34:44 Worker15 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.16:8003
(3) 18:34:44 Worker2 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.50:8005
(3) 18:34:44 Worker9 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.50:8005
(3) 18:34:44 Worker18 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.16:8003
(3) 18:34:44 Worker7 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.16:8003
(3) 18:34:44 Worker1 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.15:8003
(3) 18:34:44 Worker32 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.15:8003
(3) 18:34:44 Worker33 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.15:8003
(3) 18:34:44 Worker15 [NodeConn (acquire stream)] >> Connected: 10.2.7.16:8003
(3) 18:34:44 Worker7 [NodeConn (acquire stream)] >> Connected: 10.2.7.16:8003
(3) 18:34:44 Worker13 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.50:8005
(3) 18:34:44 Worker22 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.16:8003
(3) 18:34:44 Worker21 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.16:8003
(3) 18:34:44 Worker12 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.16:8003
(3) 18:34:44 Worker23 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.16:8003
(3) 18:34:44 Worker16 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.16:8003
(3) 18:34:44 Worker26 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.16:8003
(3) 18:34:44 Worker33 [NodeConn (acquire stream)] >> Connected: 10.2.7.15:8003
(3) 18:34:44 Worker10 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.16:8003
(3) 18:34:44 Worker30 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.15:8003
(3) 18:34:44 Worker21 [NodeConn (acquire stream)] >> Connected: 10.2.7.16:8003
(3) 18:34:44 Worker11 [NodeConn (acquire stream)] >> Connected: 10.2.7.50:8005
(3) 18:34:44 Worker31 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.15:8003
(3) 18:34:44 Worker16 [NodeConn (acquire stream)] >> Connected: 10.2.7.16:8003
(3) 18:34:44 Worker12 [NodeConn (acquire stream)] >> Connected: 10.2.7.16:8003
(3) 18:34:44 Worker9 [NodeConn (acquire stream)] >> Connected: 10.2.7.50:8005
(3) 18:34:44 Worker26 [NodeConn (acquire stream)] >> Connected: 10.2.7.16:8003
(3) 18:34:44 Worker35 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.15:8003
(3) 18:34:44 Worker30 [NodeConn (acquire stream)] >> Connected: 10.2.7.15:8003
(3) 18:34:44 Worker28 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.15:8003
(3) 18:34:44 Worker36 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.15:8003
(3) 18:34:44 Worker2 [NodeConn (acquire stream)] >> Connected: 10.2.7.50:8005
(3) 18:34:44 Worker10 [NodeConn (acquire stream)] >> Connected: 10.2.7.16:8003
(3) 18:34:44 Worker19 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.15:8003
(3) 18:34:44 Worker18 [NodeConn (acquire stream)] >> Connected: 10.2.7.16:8003
(3) 18:34:44 Worker1 [NodeConn (acquire stream)] >> Connected: 10.2.7.15:8003
(3) 18:34:44 Worker3 [NodeConn (acquire stream)] >> Connected: 10.2.7.50:8005
(3) 18:34:44 Worker32 [NodeConn (acquire stream)] >> Connected: 10.2.7.15:8003
(3) 18:34:44 Worker28 [NodeConn (acquire stream)] >> Connected: 10.2.7.15:8003
(3) 18:34:44 Worker22 [NodeConn (acquire stream)] >> Connected: 10.2.7.16:8003
(3) 18:34:44 Worker23 [NodeConn (acquire stream)] >> Connected: 10.2.7.16:8003
(3) 18:34:44 Worker36 [NodeConn (acquire stream)] >> Connected: 10.2.7.15:8003
(3) 18:34:44 Worker6 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.15:8003
(3) 18:34:44 Worker4 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.15:8003
(3) 18:34:44 Worker19 [NodeConn (acquire stream)] >> Connected: 10.2.7.15:8003
(3) 18:34:44 Worker17 [NodeConn (acquire stream)] >> Establishing new TCP connection to: 10.2.7.16:8003
(3) 18:34:44 Worker35 [NodeConn (acquire stream)] >> Connected: 10.2.7.15:8003
(3) 18:34:44 Worker13 [NodeConn (acquire stream)] >> Connected: 10.2.7.50:8005
(3) 18:34:44 Worker31 [NodeConn (acquire stream)] >> Connected: 10.2.7.15:8003
(3) 18:34:44 Worker6 [NodeConn (acquire stream)] >> Connected: 10.2.7.15:8003
(3) 18:34:44 Worker17 [NodeConn (acquire stream)] >> Connected: 10.2.7.16:8003
(3) 18:34:44 Worker4 [NodeConn (acquire stream)] >> Connected: 10.2.7.15:8003
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4304EAB-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4319174-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 433350A-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 44D31C9-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4558505-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 456B64A-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 48F80A2-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 48FDA30-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 490452B-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4905BDB-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4B0E0E0-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4BE2455-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4C12334-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4C49516-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4C58A8F-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4C84660-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4C9077A-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4CB9EFD-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4CFACDB-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 62BB077-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 62BC3DD-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4C0E04A-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4C14C78-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4C50E5F-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4C7EADC-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4C88B9B-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> Failed to insert chunk with ID 4CB5ABC-5209640D-1E3D from target 57499
(0) 21:41:05 Worker19 [RetrieveChunksWork] >> SQLite Error was error code 19
(0) 21:41:05 Worker19 [App (component exception handler)] >> This component encountered an unrecoverable error. [SysErr: No such file or directory] Exception message: std::exception
(2) 21:41:05 Worker19 [App (component exception handler)] >> Shutting down...
(0) 21:42:55 Main [ModeCheckFS] >> An error occured while fetching data from servers. Fsck cannot proceed. Please see log file for more information
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.15:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.15:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.15:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.15:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.15:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.15:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.15:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.15:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.15:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.15:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.15:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.15:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.16:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.16:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.16:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.16:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.16:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.16:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.16:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.16:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.16:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.16:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.16:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.16:8003
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.50:8005
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.50:8005
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.50:8005
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.50:8005
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.50:8005
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.50:8005
(3) 21:42:55 Main [NodeConn (invalidate stream)] >> Disconnected: 10.2.7.50:8008