Monday morning madness
So we have our fatal failure of the week, our internal server's
filesystem completely shit the bed. While I'm not going to point fingers
I will outline what exactly went wrong:
The server ran a synapse (Matrix) server.
It also hosted (static) git repositories.
Somebody was transfering a large quantity of large files while Synapse
was very, very busy doing its thing.
The health of the drive was deteriorating as early as last week.
We started unplugging other drives to back them up and as we were about
to make a copy of the main drive today, somebody wrote to the main drive
again - ultimately killing it for good.
We didn't lose any real data when warning signs of slow transfers were shown
last week.
I wasn't involved in this but I did warn about it.
So lesson learned: if your damn server is showing warning signs of slow
disk operations and stalling transfers, kill sshd and cohorts like rpc/nfsd
and don't let anyone near them, blank RSA access keys so someone can't just
scp files without your constent and mess everything up in the process.
We will also be splitting up our Matrix instance from our development system to make sure this won't have major consequences.
I made regular backups of important data, but the main-HDD with the current
Matrix setup was corrupted before we could make any serious backup.
The instance wasn't that old anyway, but I have to recreate a space or two.
TODO
----
Nuclide/FreeHL game tat:
- Gunman weapons need A LOT of work
- Team Fortress support is laughable in the weapons department, core ents
needs more testing (2fort works perfectly last time I tried)
- Poke646 is practically on halt until the other two are done
- They Hunger relies on mostly Half-Life entity crap, weapons are all
done as far as I'm told
Counter-Strike still has animation oddities I need to look at with Spoike.
It's related to how we compose the animation blends in-engine. The basebone
field manipulation crap kinda sucks. You shoot a gun while crouch and your legs
which are _NOT_ supposed to move, bob back and forth. What's going on there?
-- Marco