[NA] I'm from I.T. and I'm here to help....

[CJones]

After waiting literally 20 minutes for my computer to boot up to the point I could use it due to running start-up and logon scripts, my computer was laggy for the first hour of use.

Now I get this pretty little box on my screen that I can't close:

So.. After reduced productivity for the first hour and half of the day, my number one priority is now to reboot and go through the whole process AGAIN, because (I'm guessing) they have decided to install a new font in MS Word or something 'mission critical' like that.

When will corporate I.T. departments get it through their heads that they are should be a tool to INCREASE productivity. It seems like they have gone off in their own realm where they neither listen to, nor support operations and exists solely for their own self-proclaimed glory.

ARGH!

 

[TMetzinger]

Well, not all system patches can be installed in an unattended fashion, particularly if the system isn't up and running.

We ask users to leave their workstations on (signed off) during the overnight hours so we can push out patches and reboot as needed. Those that do have a workstation ready to go in the morning. Those that don't.... get to be interrupted the next day.

And it's not because we WANT to push out patches all the time, it's because if we don't we will end up violating a service level agreement driven by company policies - which state that patches will be installed on production systems within one month of their release.

 

[flygirl34q]

Well, not all system patches can be installed in an unattended fashion, particularly if the system isn't up and running.

We ask users to leave their workstations on (signed off) during the overnight hours so we can push out patches and reboot as needed. Those that do have a workstation ready to go in the morning. Those that don't.... get to be interrupted the next day.

And it's not because we WANT to push out patches all the time, it's because if we don't we will end up violating a service level agreement driven by company policies - which state that patches will be installed on production systems within one month of their release.

That's great for businesses that don't operate 24/7... but what about those that do?? Uggh... waay too much computer drama two nights ago when the Geek Squad decided to take our network system down to work on it... right in the middle of me working! I really wish they wouldn't do that... or else give me a time when this is going to happen, so I can be better prepared for it. It finally came back up around 4:30 or 5AM... must have been a very complicated "patch" they were pushing this time.

 

[ScottM]

I am very fortunate to work with people from many different companies. When we are not beating each other up over things that will affect our business we sit back, have a beer or really nice Scotch and discuss the world. While we all differ on politics, religion, business needs, best TV shows, etc. There is one truth that we all universally acknowledge. That is no matter who you work for IT SUX! The only guys happy with their IT departments are the consultants who do it all themselves for their one person businesses.

 

[TMetzinger]

That's great for businesses that don't operate 24/7... but what about those that do?? Uggh... waay too much computer drama two nights ago when the Geek Squad decided to take our network system down to work on it... right in the middle of me working! I really wish they wouldn't do that... or else give me a time when this is going to happen, so I can be better prepared for it. It finally came back up around 4:30 or 5AM... must have been a very complicated "patch" they were pushing this time.

Well for those systems we either:
Schedule downtime and publish it in advance.
Have redundant systems and patch the backup system, fail the application over to the backup system, patch the primary system, and fail the application back to the primary.

Bottom line is that sooner or later every computer needs patching, and that means that device is gonna be unusable for a period of time. Good communication minimizes the effects of the required downtime.

 

[flygirl34q]

Well for those systems we either:
Schedule downtime and publish it in advance.
Have redundant systems and patch the backup system, fail the application over to the backup system, patch the primary system, and fail the application back to the primary.

Bottom line is that sooner or later every computer needs patching, and that means that device is gonna be unusable for a period of time. Good communication minimizes the effects of the required downtime.

Still doesn't make it suck any less. I work alone at night in a place with 4 computers in various places and when that wonderful window came up on the computer I was working on informing me that the network was now unusable, I could take a step backwards and watch them all go down one by one. Understand that overnights is the best and least busy time to get these things accomplished, but really wish there was some way to work on them one at a time... I'd be willing to move around from one station to another (there's 4 of them and only one of me!) rather than having to wait until they all come back up and then bust my ass trying to get everything I need to done before the first day shift comes in and things start to ramp up all over again. Sigh... but I know that someone is going to say how that isn't possible because of ---------. At least I got to vent about it! Thanks anyways.

 

[murphey]

That's great for businesses that don't operate 24/7... but what about those that do?? Uggh... waay too much computer drama two nights ago when the Geek Squad decided to take our network system down to work on it... right in the middle of me working!

That's unacceptable except in extreme emergencies. If this is scheduled outage, then it should be announced a day or two in advance. Then starting about 8 hours from the outage, a message at 8, 4, 2 & one hour in advance. Then at 30 and 10 min prior to outage.

The data center manager should be made aware of "customer service" concepts, that you are a customer. Not only that, but when you are interrupted without advanced warning it impacts the entire organization in $$$$$.

 

[wsuffa]

Don't even get me started.....

Some of our folks are still royally ****ed at the way IT handled the RSA debacoe. It took 2 weeks to clean up the mess.

 

[TMetzinger]

Still doesn't make it suck any less. I work alone at night in a place with 4 computers in various places and when that wonderful window came up on the computer I was working on informing me that the network was now unusable, I could take a step backwards and watch them all go down one by one. Understand that overnights is the best and least busy time to get these things accomplished, but really wish there was some way to work on them one at a time... I'd be willing to move around from one station to another (there's 4 of them and only one of me!) rather than having to wait until they all come back up and then bust my ass trying to get everything I need to done before the first day shift comes in and things start to ramp up all over again. Sigh... but I know that someone is going to say how that isn't possible because of ---------. At least I got to vent about it! Thanks anyways.

I sympathize, and someone somewhere above you screwed up... You should have already known that was going to happen, in time for you to replan your shift around it so you could still be productive.

 

[Steve Foley]

they have decided to install a new font in MS Word or something 'mission critical' like that.

What likely happened is that one day, the VP of something-or-other received a PDF with a font that wasn't installed on his machine. This lead to a corporate mandate all machines must have the same fonts installed.

So this weekend, engineering got a drawing from SouthEast Asia for the new dingle-widget that will save the company's financial future (unlike the previous eight dingle-widgets, that failed to take off as expected).

So Engineering "Had to" have the new swizzle-stick font installed in order to properly manipulate the new dingle-widget drawing, and due to the corporate mandate of the VP of something-or-other, an emergency patch was pushed to all machines to install the new swizzle-stick font.

Sorry for the inconvenience.

 

[Ghery]

IT departments long to return to the days of the mainframe when we submitted jobs on punch cards and they ran them when they had time. Mere users were not allowed in the "holy of holies" (machine room) and were kept at arms length. After 30+ years the IT folks are still trying to strangle the PC. How dare we want to do things on our own schedule, in our own way? The nerve!

 

[Steve Foley]

Mere users were not allowed in the "holy of holies" (machine room) and were kept at arms length.

And we never spent three weeks diagnosing a problem only to find out the problem was caused when the user installed a new video driver he downloaded so he could play Doom 2012.

 

[JeffDG]

IT departments long to return to the days of the mainframe when we submitted jobs on punch cards and they ran them when they had time. Mere users were not allowed in the "holy of holies" (machine room) and were kept at arms length. After 30+ years the IT folks are still trying to strangle the PC. How dare we want to do things on our own schedule, in our own way? The nerve!

Most of the time, we in IT are mandated to do certain things, often by our security folks...that's where almost all patch pushes come from.

We generally push patches at 3am, but if someone is logged onto the PC, we give them the opportunity to delay patch installation for up to 24 hours on workstations (servers never auto-patch).

So, when you go home for the day, or night for that matter if you're a night-shift person, it'll install the patch while you're gone.

There are times, for stuff like zero-day exploits, particularly when they've been detected in our environment, that we'll push an "install right the **** now" patch, because the productivity lost being widely infected will be far more than people having to stop and go get a coffee right now, but those are quite rare, and dictated WAY above my paygrade, with the price of admission to order that being having the word "Chief" and "Officer" in your title.

 

[wsuffa]

IT departments long to return to the days of the mainframe when we submitted jobs on punch cards and they ran them when they had time. Mere users were not allowed in the "holy of holies" (machine room) and were kept at arms length. After 30+ years the IT folks are still trying to strangle the PC. How dare we want to do things on our own schedule, in our own way? The nerve!

I call those the "Ivory Tower" days.

And we never spent three weeks diagnosing a problem only to find out the problem was caused when the user installed a new video driver he downloaded so he could play Doom 2012.

You mean you still allow users to install programs or drivers without IT approval? Many companies have locked down the computers.

 

[JeffDG]

You mean you still allow users to install programs or drivers without IT approval? Many companies have locked down the computers.

That depends. Companies that want to run their IT departments with the minimum of people lock their computers down. Allowing users to install software is the source of 80+% of problems with PCs. So the beancounters can save a ton of money with a lockdown.

 

[zaitcev]

That depends. Companies that want to run their IT departments with the minimum of people lock their computers down. Allowing users to install software is the source of 80+% of problems with PCs. So the beancounters can save a ton of money with a lockdown.

We have something called "Corporate Standard Build", which either comes with a lockdown or not, depending on the user's role. In practice it relies on users' goodwill, as the CSB hardware does not include TXT or such. I don't even use CSB, just got the standard hardware and have my own software on it.

 

[wsuffa]

That depends. Companies that want to run their IT departments with the minimum of people lock their computers down. Allowing users to install software is the source of 80+% of problems with PCs. So the beancounters can save a ton of money with a lockdown.

We have a sizable IT group, and contract out some services.

I have one of the last computers that *wasn't* locked down, but doesn't matter, they installed data leak, keystroke monitoring, and periodic screen shot monitoring on all computers. Even if your computer isn't locked down, it's monitored.

Rationale is licensing compliance, preventing data leaks (thumb drives and CD/DVD writers are monitored), as well as security are the rationale.

Saps a LOT of productivity.

 

[JeffDG]

We have a sizable IT group, and contract out some services.

I have one of the last computers that *wasn't* locked down, but doesn't matter, they installed data leak, keystroke monitoring, and periodic screen shot monitoring on all computers. Even if your computer isn't locked down, it's monitored.

Rationale is licensing compliance, preventing data leaks (thumb drives and CD/DVD writers are monitored), as well as security are the rationale.

Saps a LOT of productivity.

You obviously don't have European operations...I do license compliance, and I'm not permitted, for example, to monitor how often something like "Visio" is used on a PC in Germany...it violates the user's privacy!

So, in the US, I can tell management "OK, instead of buying this software for this new project, there are 50 people with it installed that haven't run it in the last 6 months.", but in Europe...nooooo...

I think screencaps and keyloggers are just way over the top in terms of monitoring...but execution monitoring can save you a boatload of money if used effectively...you're not using it to say "Hey, that slacker never does any work!", you're using it to say "Hey, Bob just isn't using this software, let's let Doug have it instead..."

And if you can find some "license compliance" reason for doing screencaps and keylogging, I'd love to hear it, because I've seen nothing non-Orwellian justifying those.

If you don't want data leaks, block USB mass storage and CD/DVD writers (don't know the last time I saw a floppy drive).

 

[jesse]

The problem is that most of this just wastes time, makes IT people feel important, kills productivity, while costing the company a lot of money.

You can lock down things to "prevent data leaks" but any user with half a brain that wants to steal the data will steal the data.

 

[Bob Noel]

That depends. Companies that want to run their IT departments with the minimum of people lock their computers down. Allowing users to install software is the source of 80+% of problems with PCs. So the beancounters can save a ton of money with a lockdown.

How much does the lost productivity cost?

I bet that is something the beancounters ignore because they can't figure it out.

 

[alaskaflyer]

When will corporate I.T. departments get it through their heads that they are should be a tool to INCREASE productivity. It seems like they have gone off in their own realm where they neither listen to, nor support operations and exists solely for their own self-proclaimed glory.

ARGH!

That is an issue with admin "support" generally and not just IT. And it certainly is not confined to the corporate world. I once graphed out "Richard's Rule", describing the tendency for "support" people to end up in control of any organization, given enough time.

We just completed transitioning to one of those cyclical centralization efforts for human resources (oops, "workforce management".) Now all personnel actions must be submitted a full month before they are to take effect. Four years from now someone will decide this is not meeting customer needs (like they care) and we will be back (again) to embedded HR support, It has changed back and forth every 4-5 years in the past two decades. Rinse repeat. Unfortunately I will still be around when they once again move back to a centralized model.

Oh, and I read yesterday how the Department wants to suck all IT support up to the Department level By "Department" I don't mean the term in its corporate context, but rather the Department of the Interior. As in, your IT support person will be in a regional cluster somewhere.

Not everything can be accomplished via remote software.

 

[Bob Noel]

Oh, and I read yesterday how the Department wants to suck all IT support up to the Department level By "Deparrment" I don't mean the teerm in it's corporate context, but rather the Department of the Interior. As in, your IT support person will be in a regional cluster somewhere.

Not everything can be accomplished via remote software.

oh, it'll be a "cluster" all right... (>-{

 

[Steve Foley]

I call those the "Ivory Tower" days.



You mean you still allow users to install programs or drivers without IT approval? Many companies have locked down the computers.

Sure.

If there are any problems, I copy the documents folder to the network, reload the OS and software, and restore the documents.

 

[CJones]

How much does the lost productivity cost?

Let's say you run a weekly process that consumes 20% of a given computer/server processor capacity. That process runs for 4 hours one day every week.

Obviously you aren't running at 100% of processor capacity, but let's say it still decreases overall productivity by 10% for those 4 hours.

1,000 operators at, say $15/hr, operating at 90% capacity. Normally, you would get 1,000 operators * $15/hr * 100% productivity * 4 hrs = $60,000 worth of labor value. But we're operating at 90% which brings us down to $54,000 worth of labor value. So that weekly process is costing $4,000 per 1,000 operators per week in decreased productivity.

$4,000 * 52 = $208,000 worth of lost productivity for one year. That's based on 1,000 operators. Roll it up to 5,000 operators and you just cost the company $1,040,000 in lost productivity. That ain't exactly chump change...

 

[denverpilot]

Trust me, there's no "glory" in IT these days.

It looks to me like you had 235 minutes to do whatever you needed to do prior to being "forced" to reboot. The little box could just be shoved off the screen...

Pretty sure your "productivity" could have taken a break sometime in the next 4 hours... wait until lunchtime, hit the reboot button, walk away...?

 

[flygirl34q]

I sympathize, and someone somewhere above you screwed up... You should have already known that was going to happen, in time for you to replan your shift around it so you could still be productive.

Guess the really important folks in the ivory towers think that not enough goes on at night for any advance planning to be required. And, in certain situations that I run into, no amount of advance planning would have helped fix the situation anyway. The worst part is when customers stare at you when you tell them you can't bill their insurance or look up their records from a different location because the computers are taking a holiday. It's odd, but the screens still look deceptively normal even when the network is down. Just can't make everyone (anyone) happy no matter how hard you try.

 

[flygirl34q]

Trust me, there's no "glory" in IT these days.

It looks to me like you had 235 minutes to do whatever you needed to do prior to being "forced" to reboot. The little box could just be shoved off the screen...

Pretty sure your "productivity" could have taken a break sometime in the next 4 hours... wait until lunchtime, hit the reboot button, walk away...?

I was down for about 3-4 hours in my case... I ended up making popcorn.

 

[jesse]

For all you know Lynn it wasn't scheduled maintenance and it was some sort of failure that took awhile to repair. It happens

 

[denverpilot]

I must admit it's great to NOT be a "Windows guy" in the I.T. world. My servers are handling the stuff "behind the scenes" and are redundant at multiple physical locations.

If one of my boxes is down, we're running on the backup and our biggest worry is bandwidth to the redundant site and/or horsepower to "un-split" a load-balanced setup.

For end-users, the desktop OS is what we would call "a single point of failure" and we'd be trying to figure out a way to make it redundant.

 

[denverpilot]

P.S. With that said I left the office 15 minutes ago and grabbed a sandwich on the way to the data center to swap a dead hard disk out of a flaky RAID Array in a Dell PowerEdge 2950 server.

I spent much of the day patching BIOS, RAID controller firmware, backplane firmware, diagnostic tools, and lord knows what else to try to stop it from randomly kicking perfectly good hard drives out of the RAID pool.

It's the "off-line/backup server". If the on-line were to hiccup right now, we'd be in for a long night getting one or the other stable enough to do biz.

Thus, the need to go beat on it "right-now" is just as urgent as if it were handling real business traffic right now.

Flaky little beasts. I do miss Sun hardware, which rarely did stupid stuff like this, but not the price tag.

But at least no users are currently affected. Only people I'm annoying is the main help desk by cluttering up their list of current alarms until it's "happy" again.

 

[BillTIZ]

And we never spent three weeks diagnosing a problem only to find out the problem was caused when the user installed a new video driver he downloaded so he could play Doom 2012.

Your system is not locked down enough to not permit the user to change drivers or configuration?
And you allow gaming on a company machine?

Shame, Shame, Shame

 

[zaitcev]

Your system is not locked down enough to not permit the user to change drivers or configuration? And you allow gaming on a company machine?
Shame, Shame, Shame

The time for this finger-wagging has passed with the 20th century. Companies move on the wide front to erase the boundary between the work and home. Anyone who's one step above retail is expected to work essentially all the time. My CEO did not mind me napping at work even back in 1990s, and gaming is just a given for everyone. These days they'll even buy you a computer with 3D _and_ pay for a broadband wireless, just so your manager can call you for a chat during those gaming sessions.

-- Pete

 

[wsuffa]

The time for this finger-wagging has passed with the 20th century. Companies move on the wide front to erase the boundary between the work and home. Anyone who's one step above retail is expected to work essentially all the time. My CEO did not mind me napping at work even back in 1990s, and gaming is just a given for everyone. These days they'll even buy you a computer with 3D _and_ pay for a broadband wireless, just so your manager can call you for a chat during those gaming sessions.

-- Pete

That all may be true, but there are some companies that still treat employees like chattel.

 

[Bob Noel]

That all may be true, but there are some companies that still treat employees like chattel.

what more would you expect when they are stuffed into fabric covered boxes?

 

[CJones]

It looks to me like you had 235 minutes to do whatever you needed to do prior to being "forced" to reboot. The little box could just be shoved off the screen...

Nope. No minimize button. No 'Remind me Later'. And it was "Always On Top", so I couldn't put it behind the other windows. Oh, and the box came up after I had already gotten knee-deep in a project, which meant closing everything down reopening after reboot.

This 'little box' incident was just the final straw in an ongoing headache with many different aspects of I.T. I didn't mean to open up Pandora's box of I.T. bashing. Apparently lots of people have suppressed issues with I.T.

 

[wsuffa]

One IT group had a slogan: "IT - changing the way business is done"....

 

[JeffDG]

Nope. No minimize button. No 'Remind me Later'. And it was "Always On Top", so I couldn't put it behind the other windows. Oh, and the box came up after I had already gotten knee-deep in a project, which meant closing everything down reopening after reboot.

This 'little box' incident was just the final straw in an ongoing headache with many different aspects of I.T. I didn't mean to open up Pandora's box of I.T. bashing. Apparently lots of people have suppressed issues with I.T.

You need a new IT Manager or CIO.

That type of interruption of work is completely unacceptable IMHO.

 

[ScottM]

Nope. No minimize button. No 'Remind me Later'. And it was "Always On Top", so I couldn't put it behind the other windows. Oh, and the box came up after I had already gotten knee-deep in a project, which meant closing everything down reopening after reboot.

This 'little box' incident was just the final straw in an ongoing headache with many different aspects of I.T. I didn't mean to open up Pandora's box of I.T. bashing. Apparently lots of people have suppressed issues with I.T.

My old company was famous for those. Since my computer had to hook up to other networks on a regular basis and because we had confirmed attempts at being hacked from competitors on those networks we were allowed to run extra security that allowed us to block a lot of stuff. I set up my computer to also block all the probs from IT and I put an end to those annoying pop up things. OF course I had to then do manual updates to my computer, but at least I could do those and do them when it was convenient to me!

My current company has really locked down employee computers. There is so little latitude in what we can run that our computers are useless. For instance we are only able to IE version 6. Cannot even put version 8 on them, IT will pull it off. So what many of us have done is to buy our own PCs and use them for all work except the few things like expense reports that have to be run from inside the company network. The locked down IT policy has caused most company information to run on non-company assets by not serving the customer aka the employee. Hopefully we will not have an event that will cause IT to close the loopholes that let us get away from getting our email from outside of the firewall very easily.

 

[denverpilot]

Apparently lots of people have suppressed issues with I.T.

This is usually a sign that company management has no idea how to communicate their goals/requirements of IT to the rest of their staff, honestly.

If IT was "under orders" to do what they did to you, your manager should have told you it was coming.

If they were under "standing orders from higher command", they should have told your managers it was coming... etc etc etc.

Your company laptop should be being treated by IT and Management like it's a PRODUCTION system. Down-time should be scheduled in a maintenance window, etc.

 

[jason]

So, who gets blamed when your systems are left unpatched and you click on something you're not supposed to?