Wednesday, March 4, 2009

The Weekly Moan: There May Be Trouble Ahead

“Tick, Tick, Tick, Tick....Boom”

Perhaps I’m just paranoid. It’s happened before, I’ve worried unjustifiably over something happening or not happening and when that point comes, the event passes by without a hint of trouble. But right now, I’m worried.

Perhaps my worry is wrongly placed, maybe I’m getting above my station, but I’m certainly worried. It’s clear I am, I’ve spent the last few weeks blogging and complaining my way through these issues. Here’s why I’m worried;

I was given a project last year at work to upgrade to the latest anti-virus suite, a project which was rushed through from start to finish, much against my better judgement, but my manager demanded it of me and I stepped up to the plate and delivered the deployment of the new solution on time. As you’ll know from previous posts, it didn’t work and it’s caused me nothing but problems ever since trying to regain control of the anti-virus security. A security that wasn’t adequately in place before I started my project I might add.

My manager had previously neglected to upgrade from version 8 of the anti-virus suite to neither version 9, version 10 nor even version 11 of the suite, making the upgrade process to version 12 a almost impossible task of performing smoothly. My fears were realised when problem after problem appeared and the new software failed to work. Even now, today as I write, our anti-virus security platform is flawed and continues to be a growing concern to me. Not to anyone else though, my manager and my peers appear to be blind to the reports I am producing and the issues highlighted.

Last week, our ability to send external email was compromised after we managed to enlist ourselves on a number of spam databases. A complete failure of our security systems you would think – not so – my manager has once again patched up the problem and moved on, already forgetting about the incident.

My latest project, which to those who have been following my Twitter feed, is to deploy a new web filtering solution to the business. The project has had its complications, but I’ve managed to work my way through these and set up a semi-working filtering solution that we are currently putting through a very loose testing phase.

I however have spotted a problem already; it appears we have some machines on our network reporting to “malicious” websites on a continuous basis. I’ve scanned the machine for viruses and nothing has been reported, but I’m convinced that there is something there that has been compromised by the apparently lack of time investment we’ve placed into these essential security areas.

To join this issue, our backup solution has been causing endless amounts of problems over the past three weeks and we’re struggling to get a decent backup of any of our critical servers. My brief involvement with the backup solution has placed me directly into the line of fire, along with another colleague, charged with fixing the solution. The guy who actually looks after the solution on a daily basis has been off, out of the office, and unlikely to return within the next two weeks.

I’ve learnt that my colleague has no analytical skills or any logical troubleshooting skills. He has been blindly picking up error messages one after the other and fixing them, whilst simultaneously hoping and preying that any two errors are somehow related and will just “fix themselves” as a result.

I’m sorry, but I’ve been involved in this IT troubleshooting stuff for long enough now to know that you can’t analyse problems either all at once or in a fix one thing and hope the next error will magically just be also fixed – UNLESS there is a direct logical reason to believe one problem is directly impacted by another, and these can normally be proved or at least speculated to a high degree without resorting to guessing.

Earlier this week I started to write “CMA Documents” (Cover My Ass), which highlight all of the discussions, the resulting actions and the proactive tasks that I have done relating to the anti-virus platform. Today I spent my morning mapping out backup routines in order to look for patterns and correlations between backup success or failures. Patterns emerged quickly and I’ve managed to make some pretty good, logical assumptions, which I’m hoping to prove or disprove over the next few days – hopefully before we are requested to perform a data recovery we cannot possibly make.

My CMA documents have stalled; I’m not sure whether actually writing them will help me or not. My peers or my manager appear to care very little about the day-to-day operation and are instead more focused on the ‘next big thing’. We’ve rushed into virtualisation, we’ve rushed into implementing a new firewall (which we’re not even configuring ourselves and is another project that has found its way onto my desk for management). We’ve rushed into other projects too, with very little discussion or involvement from the team.

Right now, as I currently write this blog, my brain is on automatic pilot and I’m unsure where exactly to focus my attention. Security is breached, but no one cares; backups are failing, but no one is willing to listen; web filtering projects are due, and pressure is on to “just get it in”; firewalls are being replaced with no understanding or learning for the long term. It’s not how I would want to structure my IT assets or allow the IT team to be so unconcerned in their approach to what they are doing for the longer term than just a project deadline.

Perhaps I’m worried, but my mind is already else where, confined to the idea that it’s only a matter or time before it all goes wrong and fingers are pointed in my direction. My good will and effort to get the new systems in place and working far better than they have ever done in the past will be over looked and no matter how many details or how many CMA documents I produce and pass around, they will fall on deaf ears or ignored or worse, the issues passed back to me to resolve without any assistance or buy in from any other member of the team, including my manager.

These are scary times and whilst I’ve just been passed the next issue to jump to the top of my daily things to do – why a user can’t play YouTube videos – the biggest thing my boss is asking from me right now is that I complete a health and safety booklet, so that I can hand it in and be able to sign off work permits in the future. Hardly an IT procedure and hardly something that I, personally, am even remotely concerned with completing, especially when more pressing IT matters are currently circling and closing in. The words “ticking” and “time bomb” seem to be appropriate right now....

0 comments:

Template by - Abdul Munir | Daya Earth Blogger Template