Tuesday, April 4, 2017

The dreaded failed check, some nightmares are real.


Written by David Ianetta

More often than I like, I see something in an RMM dashboard that causes me to shudder; and really it should be keeping the MSP up at night.

For me, it normally pops its ugly head out unexpectedly during a routine training tech walk.

An MSP calls to get refresher on the RMM Dashboard; or perhaps they are new and I am explaining how to understand and adjust checks.

And there it is.

It’s up on the screen, for both of us to see.

The dreaded Failed Login Check!

Now, many of these are normal, no cause for real concern.

Someone forgets a server password and they bang away a few times before remembering.
Or perhaps a service was using an AD account, then that account was disabled so the service keeps trying.

It’s always best to check them out. Best to know.

That’s not what causes me the concern.

You are going to see these as a part of doing RMM all the time. What truly bothers me is when the numbers are really high.

Like, 5,000, or 10,000. Or like my sample taken from an actual dashboard 15,129 times to be exact.

When you see a number like that, there is only one real answer. I hover my mouse over the number, brace for impact and click.

Yup, there it is. A giant list of random names, all trying to log into a server. And that can only be one thing. Someone is trying to hack the server. Someone has gotten through a firewall and is now trying to get one step further into the server.

Once they gain access, it can only get worse. They can do anything from recruiting all the machines into a botnet, distributing ransom-ware or mining data. 

And here’s the even scarier part.

You see that, and you have to wonder, “Ok they failed 15,000 times… did they finally make it in?

So I’m going to share three things you need to do to keep this from catching you off guard.

First keep your dashboard clean.

When my son was in Marine boot camp he told me they had to keep their combination locks all on zero. Everything had to be exact. The thinking was you get tuned to knowing when something is out of place.

For many of you that means cleanup those checks! Adjust the performance monitoring checks, delete the drive checks that are searching for that S drive that no longer exist. When a check fails, it'll be more likely to stand out to you and get your attention as it should. Get rid of the noise.

Next login to your dashboard regularly and/or pay attention to your alert emails. Think of RMM as another tech on your team, don't ignore what it has to tell you. If a failed check is noise, adjust it or delete it. Don't just leave it there.

And finally if you do find a check like this, extra information can show you source IP information that can get you started for locking down the network. Time to tighten up security, close ports, change passwords and by all means get those machines scanned! This is a great time to finally get that customer on board with security.

As bad as you finding this in your check is, it is far worse if your customer brings it to your attention after a breach.