Today Cyber Security plays a paramount role in global security. On this blog, the CEO of Paramount Defenses shares rare insights on issues related to Cyber Security, including Privileged Access, Organizational Cyber Security, Foundational Security, Windows Security, Active Directory Security, Insider Threats and other topics.


November 5, 2018

Cyber Security 101 for the C-Suite - Active Directory Security is Paramount

Folks,

Today's post is for all executives worldwide who comprise the C-Suite at thousands of organizations worldwide.


I pen today's post with profound respect for all executives worldwide, because I understand first-hand just how important the nature of their responsibilities is, how valuable their time is, and how far-reaching the consequences of their decisions are.

A quick footnote for all C*Os : In case you're wondering who I am to be penning this, I'm former Microsoft Program Manager for Active Directory Security. Relevance? Microsoft's Active Directory is the foundation of your entire organization's cyber security. Finally, like you, I also happen to be the CEO of a $ Billion+ company.

Today's post is in the form of a simple letter, that follows (below.)

Thanks,
Sanjay


<Begin Letter>

Subject - Cyber Security 101 for the C-Suite


To: Chairmen, CEOs and CFOs Worldwide



Dear C*O,

Hi, I'm Sanjay, former Microsoft Program Manager for Active Directory Security, but more importantly a sincere well-wisher who cares deeply about cyber security, and who just happens to know a thing or two about the very technology that lies at the very foundation of cyber security of your ($ Billion to $ Trillion) organization, and those of 85% of all organizations worldwide.

I write to you to bring to your attention a matter of paramount importance to your organization's foundational security.



Context - Foundational Security

Today we all engage in business in what is essentially a global digital village, wherein just about just every aspect of business, whether it be production, marketing, sales, customer-service, collaboration, finance etc. etc. substantially relies on technology.


Within our respective organizations, it is our IT infrastructure that enables and empowers our workforce to engage in business.

For instance, we all (including us C*Os) log on to a computer every day, send and receive email, and create, share and access digital assets (e.g. documents, applications, services etc.) all of which are securely stored on our organizational computers.

It is only logical then that ensuring the security of the very IT infrastructure that enables and empowers our entire workforce to engage in business digitally, and the security of our digital assets is vital. In other words, cyber security is very important.

Now, if I told you that at the very foundation of your entire IT infrastructure, and consequently at the very foundation of the security of all your digital assets lay a single high-value asset, then I think you'd agree that its security would be paramount.

At the very foundation of your organization's IT infrastructure and that of its cyber security, and by corollary the cyber security of the entirety of all your digital assets (e.g. thousands of computers, thousands of employee user accounts and passwords, every single organizational email sent and received every minute of every day, all your applications, services, Intranet portals, Internet facing applications etc.) as well as the entirety of your organization's data, lies a single technology - Microsoft Active Directory.


Most simply put, Active Directory is the database that contains, stores and protects the entirety of your organization's building blocks of cyber security - each one of thousands of user accounts and their passwords, each one of thousands of computer accounts (for all laptops, desktops, servers etc.), each one of thousands of security groups that protect all your data etc. etc.

If your organization's Active Directory were compromised, everything would immediately be exposed to the risk of compromise.

Thus as you'll hopefully agree, ensuring the security of your organization's foundational Active Directory is well, paramount.



A Provable Concern - Inadequate Protection

Now, you might most likely be thinking - Well, if that's the case, I'm sure that our CIO, our CISO and their world-class IT and Cyber Security teams know all this, and have it adequately taken care of, so why should I be concerned ?


Here's why you should be concerned - In all likelihood, not only may your world-class IT and Cyber Security teams not have this adequately covered, they may have yet to realize just how very important, and in fact paramount Active Directory security is.

Further, they likely may not know what it actually takes to adequately secure your organization's  foundational Active Directory.

Now, as incredulous as that may sound, you have to trust me on this, not because I'm asking you to do so as a concerned well-wisher, but because I'm asking you to do so as arguably the world's #1 subject matter expert on Active Directory Security.

You see, prior to doing what I currently do, I was Microsoft's subject matter expert for Active Directory Security on Microsoft's Windows Server Development team. In case you're curious as to what I do currently do with all this knowledge, well, its this.

As the world's leading subject matter expert on Active Directory Security, I would highly encourage you to ask your IT and Cyber Security leadership, specifically your CIO and your CISO, just how secure they think your organization's Active Directory is.



Simple Proof - You Just Have to Ask

When you ask them about it, please do request specific answers, and here are 7 simple questions you can ask them, the answers to which will give you an indication of just how secure your organization's Active Directory actually is today -


  1. Is the security of our foundational Active Directory deployment a top cyber security priority today?

  2. Do we know exactly what the Top-5 security risks to our foundational Active Directory are?

  3. Do our Active Directory Admins know what Active Directory Effective Permissions are?

  4. Do we know exactly who possesses what level of privileged access in our Active Directory?

  5. Do we know exactly who can control/manage each one of our Active Directory privileged accounts and groups?

  6. Do we know exactly who can run Mimikatz DCSync against our Active Directory today?

  7. Can you tell me exactly who can reset my domain user account's password to then be able to login as me?

I could suggest 50 such elemental cyber security questions, but for now these 7 simple, precise questions will suffice as there are only 2 possibilities here - either your IT and cyber security leadership have exact answers to these questions, or they don't.


If they can't give you exact answers to these questions, your organization's Active Directory is not secure - its as simple as that.


They might tell you that this is complicated or that they have a good approximation, or that this is very difficult to do, or that they have many other latest buzzword measures like Active Directory Auditing, Privileged Access Management, ATA, Just-in-Time Administration etc. in place, but none of that matters, because the truth is simple - they either have exact answers, or they don't.

(These questions are paramount to cyber security, and today there exists technology that can enable every organization in the world to answer them precisely, but because Microsoft likely forgot to adequately educate its customers, your IT personnel may likely not even know the importance of these paramount questions, let alone knowing what it takes to correctly answer them.)

If a $Billion+ organization doesn't even know exactly who has what privileged access in their Active Directory, as well as exactly who can manage each one of their privileged accounts and groups, how could their Active Directory possibly be secure?

If an organization's foundational Active Directory is not secure, how can the entirety of the organization's digital (IT) assets be secure, and if that's not case, how could an organization possibly be considered secure from a cyber security perspective?



Driving Change

As a member of the C-Suite, you not only have the privilege of being able to impact vital change in your organization, you also have the responsibility and the authority to demand and ensure the cyber security of the very foundation of your organization.


As a C*O, one of the most important responsibilities you shoulder is ensuring that your organization is secure, and ensuring that the very foundation of your organization's IT infrastructure and cyber security are always adequately protected, is paramount.




The Likely Reason (Optional Reading)

Here's the likely reason for why such a common-sense yet paramount matter may not be on your CIO's and CISO's radar yet.

You see, your CIO and CISO shoulder great responsibility. Unfortunately, amongst many other things, they're likely also being guided by inputs from a 1000 cyber security companies, who unfortunately may not be the best source of objective guidance.

For instance, consider CyberArk, a highly respected $ Billion+ cyber security company, that claims that over 50% of the Fortune 100's CISOs rely on them. As a subject matter expert, I can tell you that CyberArk itself may not know how to correctly assess privileged access in an Active Directory, so you see, unfortunately your CIO and CISO may not be getting the best guidance.

CyberArk is absolutely correct that "Privilege is Everywhere." However, those who know Windows Security will tell you that in a Windows network powered by Active Directory, the majority of all privileged access (delegated & unrestricted) lies inside Active Directory, but CyberArk doesn't seem to have the capability to correctly audit privileged access inside Active Directory.


The majority of all Privileged Access,including the "Keys to the Kingdom", resides inside Active Directory

CyberArk isn't alone. As unbelievable as it may sound, today even Microsoft doesn't seem to know what it takes to do so, let alone possessing the capability to help its customers correctly do so. In fact, most of the world's top IT Consulting, Audit, Cloud and Cyber Security companies also operate on Active Directory, and they too likely have neither a clue nor the capability to accurately determine exactly who has what privileged access in their own foundational Active Directory deployments.

You may find this hard to believe, but of the 1000+ cyber security companies exhibiting or presenting at the upcoming RSA Conference 2019, not a single one of them can help your organization's IT personnel fulfill such a fundamental yet paramount cyber security need - finding out exactly who has what privileged access in your organization's foundational Active Directory.

In their defense, I'll say this - if it were easy, they would've all done it by now. Unfortunately, as paramount as it is, its not easy.

Thus, I know what your CIO and CISO may perhaps not yet know, or understand the paramount importance of, which is that of all the things that need to be secured, none could possibly be more important than securing your organization's foundational Active Directory, so I thought I'd share this with you, because as a member of the C-Suite, you could provide them strategic guidance and the executive support that their teams need to accomplish this paramount objective for your organization.



In Conclusion

I only wrote this letter because we're all in this together, and I care deeply about foundational cyber security, as hopefully do you, and I felt that I could perhaps help bridge the gap between those tasked with the great responsibility of securing Active Directory (i.e. your IT personnel) and those whose executive support they need to be able to do so (i.e. you, the C-Suite.)

If any of what I shared above made sense, I would encourage you to embrace my suggestions earnestly, and act upon them, and if needed, I can prove and demonstrate every thing I've shared above, and you should feel free to take me up on this.

As for myself, all I can say is that today my work and knowledge silently help secure and defend so many of the world's most important organizations across six continents worldwide.

Thank you for your time.

Respectfully,
Sanjay Tandon.

Chairman and CEO,
Paramount Defenses



PS: Please know that I am also doing my bit to help Microsoft and the World better Understand Active Directory Security



<End Letter>

October 28, 2018

Happy Birthday, Bill Gates!

Dear Bill,

Here's wishing you Sir, likely the most successful and influential person of not just our time, but of all time, a very Happy B'day!

Photo source and attirbution: https://mobile.twitter.com/BillGates/photo

Most of the world knows you as the Founder of Microsoft, a great philanthropist, and the world's wealthiest* person.


Based on my personal experience, I however know you to be someone who truly exemplifies the very words I strive to live by, and ideally, that we should all strive to live by, because in the grand scheme of things, we are all here for relatively little time.


Deep Gratitude for Mr. Gates

If I may, I'd like to share from my personal experience, a very small example of Mr. Gates thoughtfulness, humility and kindness.

One day back in 2004, when I was a Microsoft employee, I got a call from the Reception of Building 33, the Executive Building at Microsoft, and I was asked to come and pick something up - when I reached there, the kind lady at the reception gave me a package and said that "Bill left this for you, as he's unfortunately out of town today," and in it was a note written by Bill himself - "To Sanjay, Happy Birthday, Bill Gates" ( here.) (BTW, this is not customary at all at Microsoft; in fact, it was an absolute rarity.)

I couldn't believe it. Bill Gates, our CEO, and the world's most successful and wealthiest person, made and took the time to wish me Happy B'day, and since he was going to be out of town, he was thoughtful enough to have it be given to me on my b'day!

Since that day, for the last fourteen years I've been working tirelessly to be able to express my profound respect and gratitude to Mr. Gates, and it is for the first time, that I feel I've done my bit to be able to thank him, not just in words, but in global IMPACT.


Mr. Gates, it is your greatness, kindness and humility that inspired me to conquer proverbial mountains as I persevered against all odds to ultimately build and deliver a paramount capability needed to secure and defend the very foundation of cyber security of and across Microsoft's global organizational customer base i.e. your one little act of kindness, led to and inspired THIS.



Birthday Wishes

Mr. Gates, today, you're wished profound joy and excellent health, but above all, you're wished that which is a rarity today, and that which sometimes even all the money in the world can't buy - True Peace of Mind and Happiness in the Simplest of Things!


BillG, I thank you for the incredible human being you are, and wish you a truly wonderful year ahead.

Namaste,
Sanjay.


PS: I occasionally come across monetarily wealthy people, you know, little multi-millionaires and billionaires, and some of them exude such arrogance, that I feel like telling them that there are people out there (e.g. you) who could buy all their wealth out a hundred times over, so how about a little humility?! :-) In stark contrast, I visited the Gates Foundation website today, and it was so incredibly refreshing to see it unequivocally communicate that All Lives Have Equal Value!  You Sir, command my respect.

October 27, 2018

Words I Live By

Folks,

Today, I just wanted to take a moment to share with you the words I live by -


No matter who we are, we should always strive to be ...

Sincerely,
Sanjay.

October 26, 2018

What Lies at the Foundation of Organizational Cyber Security Worldwide?

Folks,

In days to come, I'm going to answer both, the most important, and the second most important question in all of Cyber Security

Today though, I just wanted to ask a simple (rhetorical) cyber security question, so that CEOs, CIOs, CISOs and IT Directors at organizations worldwide realize just what lies at the very foundation of the cyber security of their multi-billion $ organizations.

Microsoft Active Directory

Today, at the very foundation of organizational cyber security worldwide, lie their foundational Active Directory deployments.

Consequently, it logically follows that all organizations that operate on Microsoft Active Directory are only as secure as are their foundational Active Directory deployments. After all, no matter how tall, every skyscraper is only as strong as its foundation.

In days to come, I'll share with you just how secure foundational Active Directory deployments are worldwide today - right here.

Best wishes,
Sanjay

October 13, 2018

A Very Simple Trillion $ Cyber Security Multiple-Choice Question

Folks,

In days to come, I'll be helping organizations worldwide understand what constitutes a privileged user in Active Directory, how to correctly audit privileged access in Active Directory, and what the world's most important Active Directory security capability is.

Today though, I just wanted to ask a very simple and elemental cyber security multiple-choice question, so here it is -


Q. What are the minimum Active Directory Security Permissions that a perpetrator needs to be able to successfully run Mimikatz DCSync against an organization's foundational Active Directory deployment?

Is it -
A. The "Get Replication Changes" Extended Right 
B. The "Get Replication Changes All" Extended Right 
C. Both A and B above 
D. Something else

I already know the answer to this simple question. I'm only asking because I believe that today every Domain Admin and every CISO at every organization that operates on Active Directory MUST know the answer to this question, and here's why.

You may be surprised if I were to share with you just how many Domain Admins and CISOs (at so many of the world's most prominent organizations) don't know even seem to know what Mimikatz DCSync is, let alone knowing the answer!

If you know the answer to this question, and care to share, please feel free to share it by leaving a comment below.

Best wishes,
Sanjay.

September 26, 2018

Time to Ignite An Intellectual Spark at Microsoft Ignite 2018!

Folks,

This week, thousands of IT professionals, managers, CISOs and CIOs are in Orlando, attending, well, Microsoft Ignite 2018 !

Image Courtesy Microsoft. Source: https://www.microsoft.com/en-us/ignite

Now, according to Microsoft's website, Microsoft Ignite has SOLD OUTGreat!  There are 900+ sessions, 100+ instructor-led technology workshops, 60+ Microsoft Immersion workshops, and 50+ hands-on labs with access to expert proctors!


Did I mention that of course, Microsoft's very own experts are also going to be there, and collectively, they covered numerous vital areas such as Securing the Enterprise, Simplified IT Management, Identity‚ Access & Compliance, Enterprise Security etc.


So, with over 1000 sessions, 1000s of attendees, access to "expert proctors", and 100s of Microsoft's very own IT experts, THERE MUST BE AT LEAST ONE PERSON AT MICROSOFT IGNITE who could answer A very SIMPLE QUESTION -


       Question - What's The World's Most Important Active Directory Security Capability?



Now, in case you're wondering why anyone and in fact everyone attending Microsoft Ignite should care about this question, its because in a Microsoft Windows Server based IT Infrastructure, NOT A SINGLE ONE of the numerous vital areas listed above i.e. Securing the Enterprise, Simplified IT Management, Identity‚ Access & Compliance, Enterprise Security etc. etc. can be adequately addressed without FIRST ENSURING THE SECURITY of their foundational Active Directory deployments!


Guess what?!  I'm willing to bet that 99% of experts (let alone attendees) at Microsoft Ignite don't have a clue as to the answer!


Unbelievable, haan?! So much so for a US $ 800 Billion company's  "Sold Out"  IT Conference, where 100s of world renowned IT experts, including Microsoft's finest, were presenting, and where 1000s of IT professionals (including Domain Admins of most Fortune 100 companies) were attending, yet no one likely knows the answer to this most basic of Windows Security questions!


Er, what's that millennial lingo again? Ah yes,  OMG  LOL ROFL !

Doesn't anyone RTM today?  (They don't, and here's likely why.)


On a serious note, if anyone attending Microsoft Ignite 2018 (including Microsoft's own experts) knows the answer to this 1 question, be my guest and answer the question by leaving a comment at the end of that blog post, and you'll earn my respect.


If you don't know the answer, I highly recommend reading, one, two and three, because without knowing the answer to this 1 question (and without possessing this capability,) you cannot secure anything in an Active Directory based Windows network.


Best wishes,
Sanjay

September 24, 2018

Pardon the Absence, and Get Ready!

Folks,

Hello again. I trust this finds you all doing well. It has been a few weeks since I last blogged. I hope you'll pardon my absence.

Yes I was supposed to answer a rather important question, in fact, possibly the world's most important cyber security question, for the whole world, back in July, but I had to postpone doing so, for a few good reasons, which I may reveal in days to come.

Let's just say that amongst other things (e.g. a rather interesting trip across the Atlantic), I was working on finalising a project that directly impacts cyber security worldwide today, you know, the kind of stuff that even James Bond doesn't have yet!



By the way, speaking of Mr. Bond, as you probably know, I'm a huge fan, so thought I'd share a catchy tune with you -



Oh, that project I was working is almost over (i.e. RC1), so its time for me to get back to blogging, and...     … well, get ready!

Best wishes,
Sanjay

July 9, 2018

A Trillion $ Cyber Security Question for Microsoft and CISOs Worldwide

Folks,

Today, to give a hint for the answer to this 1 question, I asked possibly the most important cyber security question in the world, one that directly impacts the foundational security of 1000s of organizations worldwide, and thus one that impacts the financial security of billions of people worldwide -


What's the World's Most Important Active Directory Security Capability?




Those who don't know why this is the world's most important cyber security question may want to connect one, two and three

I sincerely hope that someone (anyone) at Microsoft, or that some CISO (any ONE) out there, will answer this ONE question.

Best wishes,
Sanjay.

July 4, 2018

Happy 4th of July!

Folks,

Here's wishing you all a very Happy Fourth of July!  Hope you have a great one!


I was supposed to answer a certain question today, but I decided to take the day off, so I'll answer it in days to come.

Best wishes,
Sanjay.

July 3, 2018

Mimikatz DCSync Mitigation

Folks,

A few days ago I asked a (seemingly) very simple question ; no I'm not referring to this one, I'm referring to this one here  -

Can Anyone (i.e. any Cyber Security Company or Expert) Help Thousands of Microsoft's Customers MITIGATE the Risk Posed by Mimikatz DCSync?

Here's why I did so - While there's a lot of info out there on the WWW about how to use Mimikatz DCSync, and/or how to detect its use, there isn't one other* single correct piece of guidance out there on how to mitigate the risk posed by Mimkatz DCSync.

So, as promised, today I am (literally) going to show you exactly how thousands of organizations worldwide can now easily and demonstrably actually mitigate the very serious cyber security risk posed to their foundational security by Mimikatz DCSync.


In light of what I've shared below, organizations worldwide can now easily mitigate the serious risk posed by Mimikatz DCSync.




First, A Quick Overview

For those who may not know, and there are millions who don't, there are three quick things to know about Mimikatz DCSync.


Mimikatz DCSync, a Windows security tool, is the creation of the brilliant technical expertise of Mr. Benjamin Delpy, whose work over the years has very likely (caused Microsoft a lot of pain ;-) but/and) helped substantially enhance Windows Security.

Mimikatz DCSync targets an organization's foundational Active Directory domains, and instantly gives any attacker who has sufficient privileges to be able to replicate sensitive content from Active Directory, access to literally everyone's credentials!

Thus far, the only guidance out there is on how to DETECT its use, but this is one of those situations wherein if you're having to rely on detection as a security measure, then its unfortunately already TOO late, because the damage has already been done.



Detection Is Hardly Sufficient

They say a picture's worth a thousand words, so perhaps I'll paint a picture for you. Relying on detection as a security measure against Mimikatz DCSync is akin to this -

Castle romeo2

Lets say a nuclear weapon just detonated in a city, and the moment it did, detection sensors alerted the city officials about the detonation. Well, within the few seconds in which they received the alert, the whole city would've already been obliterated i.e. by the time you get the alert, literally everyone's credentials (including of all privileged users) would've already been compromised!

Make not mistake about it - a single successful use of Mimikatz DCSync against an organization's foundational Active Directory domain is tantamount to a complete forest-wide compromise, and should be considered a massive organizational cyber security breach, the only way to recover from which is to completely rebuild the entire Active Directory forest from the ground up!

This is why detection is grossly insufficient as a security measure, and what organizations need is the ability to prevent the use of Mimikatz DCSync's against their foundational Active Directory domains & thus the ability to mitigate this risk is paramount.



How to Mitigate Mimikatz DCSync

The key to mitigating this risk lies in identifying what it technically takes to be able to successfully use Mimikatz DCSync.

Specifically, if you know exactly what privileges an attacker needs to be able to successfully use Mimikatz DCSync against your Active Directory domain, then by ensuring that only highly-trustworthy, authorized individuals (and not a single other individual) actually currently possess those required privileges in your IT infrastructure, you can easily mitigate this risk.


Technically speaking, all that an attacker needs to successfully use Mimikatz DCSync is sufficient Get Replication Changes All effective permissions on the domain root object of an Active Directory domain, so all that organizations need to do is accurately identify exactly who has these effective permissions on the domain root object of each of their Active Directory domains.

While by default only the default administrative Active Directory security groups are granted this permission, since most Active Directory deployments have been around for years, and have likely gone through a substantial amount of access provisioning, in most Active Directory, a lot many more individuals than merely the members of the default AD admin groups may likely have this highly sensitive effective permission granted to them, either directly or via group membership, some of which may be direct, whilst others may be via nested group memberships, resulting in a potentially large and unknown attack surface today.

Now, it is paramount to understand ONE subtle but profound difference here - it is NOT who has what permissions on the domain root that matters, but who has what effective permissions on the domain root that matters, and this difference could be the difference between a $100 B organization being completely compromised or being completely protected from compromise.



The Key - Active Directory Effective Permissions

If you've followed what I've shared above, then you'll agree and understand that the key to being able to successfully mitigate the serious risk posed by Mimikatz DCSync lies in being able to accurately determine effective permissions in Active Directory.



In fact Effective Permissions are so important, essential and fundamental to Windows and Active Directory Security, that of the four tabs in all of Microsoft's Active Directory Management Tooling, one entire tab is dedicated to Effective Permissions.

Unfortunately, it turns out that not only is Microsoft's native Effective Permissions Tab not always accurate, it is substantially inadequate, and while I could elaborate on that, I'd rather let you come to the same conclusion yourself, and this ONE glaring inadequacy will be self-evident the moment you attempt to use it to try and find out exactly whom amongst the thousands of domain user account holders in your Active Directory domain(s), actually has the required effective permissions. In fact, the same is true of all tools/scripts that involve the use of Microsoft's APIs to do so, such as this dangerously inaccurate free tool.

Fortunately, in a world whose population is 7,000,000,000+ today, thanks to one (1) inconsequential individual, there's hope...



Finally, How to Easily and Reliably Mitigate the Risk Posed by Mimikatz DCSync

Here's a very short (and perhaps boring but insightful) video on how organizations worldwide can reliably mitigate this risk -


Note: This is NOT intended to demonstrate our unique tooling. It is solely intended to show what it takes to mitigate this serious risk. We have no particular interest in licensing our unique tooling to anyone. As such, over the years, we have NEVER, not once pitched our tooling to anyone; we've had almost 10,000 organizations worldwide knock at our doors completely unsolicited, so I hope that makes this point unequivocally.

Thus, as seen in the short video above, with the right guidance (knowledge) and capability (tooling), organizations worldwide can now easily and reliably mitigate the serious cyber security risk posed by Mimikatz DCSync to their foundational security.

Complete, illustrated, step-by-step details on how to easily and correctly mitigate Mimikatz DCSync can now be found here.


I'll say this one last time - a single successful use of Mimikatz DCSync against an organization's foundational Active Directory is tantamount to a forest-wide compromise and constitutes a massive cyber security breach, which is why mitigation is paramount.

Best wishes,
Sanjay


PS: *Here are 4 posts I've previously penned on Mimikatz DCSync - a summary, technical details, a scenario and the question.

PS2: In days to come, I'll answer this question too.

June 29, 2018

WHAT is the ONE Essential Cyber Security Capability WITHOUT which NOT a single Active Directory object or domain can be adequately secured?


Folks,

Hello again. Today onwards, as I had promised, it is finally TIME for us to help SAFEGUARD Microsoft's Global Ecosystem.


Before I share how we uniquely do so, or answer this paramount question, or ask more such ones, I thought I'd ask likely the most important question that today DIRECTLY impacts the foundational cyber security of 1000s of organizations worldwide.



Here It Is -
What Is the 1 Essential Cyber Security Capability Without Which NOT a single Active Directory object, domain, forest or deployment can be adequately secured?



A Hint

I'll give you a hint. It controls exactly who is denied and who is granted access to literally everything within Active Directory.


In fact, it comes into play every time anyone accesses anything in any Active Directory domain in any organization worldwide.




Make No Mistake

Make no mistake about it - one simply CANNOT adequately protect anything in any Active Directory WITHOUT possessing this ONE capability, and thus one simply cannot protect the very foundation of an organization's cyber security without possessing this ONE paramount cyber security capability. It unequivocally is as remarkably simple, elemental and fundamental as this.



Only 2 Kinds of Organizations

Thus, today there are only two kinds of organizations worldwide - those that possess this paramount cyber security capability, and those that don't. Those that don't possess this essential capability do not have the means to, and thus cannot adequately protect, their foundational Active Directory deployments, and thus by logic are provably and demonstrably insecure.


If you know the answer, feel free to leave a comment below.
I'll answer this question right here, likely on July 04, 2018.

Best,
Sanjay

June 18, 2018

Alarming! : Windows Update Automatically Downloaded and Installed an Untrusted Self-Signed Kernel-mode Lenovo Driver on New Surface Device

Folks,

Given what it is I do, I don't squander a minute of precious time, unless something is very important, and this is very important.


Let me explain why this is so alarming, concerning and so important to cyber security, and why at many organizations (e.g. U.S. Govt., Paramount Defenses etc.), this could've either possibly resulted in, or in itself, be considered a cyber security breach.

Disclaimer: I'm not making any value judgment about Lenovo ; I'm merely basing this on what's already been said.


As you know, Microsoft's been brazenly leaving billions of people and thousands of organizations worldwide with no real choice but to upgrade to their latest operating system, Windows 10, which albeit is far from perfect, is much better than Windows Vista, Windows 8 etc., even though Windows 10's default settings could be considered an egregious affront to Privacy.

Consequently, at Paramount Defenses, we too felt that perhaps it was time to consider moving on to Windows 10, so we too figured we'd refresh our workforce's PCs. Now, of the major choices available from amongst several reputable PC vendors out there, Microsoft's Surface was one of the top trustworthy contenders, considering that the entirety of the hardware and software was from the same vendor (, and one that was decently trustworthy (considering that most of the world is running their operating system,)) and that there seemed to be no* pre-installed drivers or software that may have been written in China, Russia etc.

Side-note: Based on information available in the public domain, in all likelihood, software written in / maintained from within Russia, may still likely be running as System on Domain Controllers within the U.S. Government.

In particular, regardless of its respected heritage, for us, Lenovo wasn't  an option, since it is partly owned by the Chinese Govt.

So we decided to consider evaluating Microsoft Surface devices and thus purchased a couple of brand-new Microsoft Surface devices from our local Microsoft Store for an initial PoC, and I decided to personally test-drive one of them -

Microsoft Surface



The very first thing we did after unsealing them, walking through the initial setup and locking down Windows 10's unacceptable default privacy settings, was to connect it to the Internet over a secure channel, and perform a Windows Update.

I should mention that there was no other device attached to this Microsoft Surface, except for a Microsoft Signature Type Cover, and in particular there were no mice of any kind, attached to this new Microsoft surface device, whether via USB or Bluetooth.


Now, you're not going to believe what happened within minutes of having clicked the Check for Updates button!



Windows Update
Downloaded and Installed an Untrusted
Self-Signed Lenovo Device Driver on Microsoft Surface! -

Within minutes, Windows Update automatically downloaded and had installed, amongst other packages (notably Surface Firmware,) an untrusted self-signed Kernel-mode device-driver, purportedly Lenovo - Keyboard, Other hardware - Lenovo Optical Mouse (HID), on this brand-new Microsoft Surface device, i.e. one signed with an untrusted WDK Test Certificate!

Here's a snapshot of Windows Update indicating that it had successfully downloaded and installed a Lenovo driver on this Surface device, and it specifically states "Lenovo - Keyboard, Other hardware - Lenovo Optical Mouse (HID)" -


We couldn't quite believe this.

How could this be possible? i.e. how could a Lenovo driver have been installed on a Microsoft  Surface device?

So we checked the Windows Update Log, and sure enough, as seen in the snapshot below, the Windows Update Log too confirmed that Windows Update had just downloaded and installed a Lenovo driver -


We wondered if there might have been any Lenovo hardware components installed on the Surface so we checked the Device Manager, and we could not find a single device that seemed to indicate the presence of any Lenovo hardware. (Later, we even took it back to the Microsoft Store, and their skilled tech personnel confirmed the same finding i.e. no Lenovo hardware on it.)

Specifically, as you can see below, we again checked the Device Manager, this time to see if it might indicate the presence of any Lenovo HID, such as a Lenovo Optical Mouse, and as you can see in the snapshot below, the only two Mice and other pointing devices installed on the system were from Microsoft - i.e. no Lenovo mouse presence indicated by Device Manager -



Next, we performed a keyword search of the Registry, and came across a suspicious Driver Package, as seen below -


It seemed suspicious to us because as can be seen in the snapshot above, all of the other legitimate driver package keys in the Registry had (as they should) three child sub-keys i.e. Configurations, Descriptors and Strings, but this specific one only had one subkey titled Properties, and when we tried to open it, we received an Access Denied message!

As you can see above, it seemed to indicate that the provider was Lenovo and that the INF file name was phidmou.inf, and the OEM path was "C:\Windows\SoftwareDistribution\Download\Install", so we looked at the file system but this path didn't seem to exist on the file-system. So we performed a simple file-system search "dir /s phidmou.*" and as seen in the snapshot below, we found one instance of such a file, located in C:\Windows\System32\DriverStore\FileRepository\.

Here's that exact location on the file-system, and as evidenced by the Created date and time for that folder, one can see that this folder (and thus all of its contents), were created on April 01, 2018 at around 1:50 am, which is just around the time the Windows Update log too confirmed that it had installed the Lenovo Driver -



When we opened that location, we found thirteen items, including six drivers -


Next, we checked the Digital Signature on one of the drivers, PELMOUSE.SYS, and we found that it was signed using a self-signed test Windows Driver certificate, i.e. the .sys files were SELF-SIGNED by a WDKTestCert and their digital signatures were NOT OK, in that they terminated in a root certificate that is not trusted by the trust provider -


Finally, when we clicked on the View Certificate button, as can be seen below, we could see that this driver was in fact merely signed by a test certificate, which is only supposed to be used for testing purposes during the creation and development of Kernel-mode drivers. Quoting from Microsoft's documentation on Driver Testing "However, eventually it will become necessary to test-sign your driver during its development, and ultimately release-sign your driver before publishing it to users." -


Clearly, the certificate seen above is NOT one that is intended to be used for release signing, yet, here we have a Kernel-mode driver downloaded by Windows Update and installed on a brand new Microsoft surface, and all its signed by is a test certificate, and who knows who wrote this driver!

Again, per Microsoft's guidelines on driver signing, which can also be found here, "After completing test signing and verifying that the driver is ready for release, the driver package has to be release signed", and AFAIK, release signing not only requires the signer to obtain and use a code-signing certificate from a code-signing CA, it also requires a cross cert issued by Microsoft.

If that is indeed the case, then a Kernel-mode driver that is not signed with a valid code-signing certificate, and one whose digital signature does not contain Microsoft's cross cert, should not even be accepted into the Windows Update catalog.

It is thus hard to believe that a Windows Kernel-Mode Driver that is merely self-signed using a test certificate would even make it into the Windows Update catalog, and further it seems that in this case, not only did it make it in, it was downloaded, and in fact successfully installed onto a system, which clearly seems highly suspicious, and is fact alarming and deeply-concerning!

How could this be? How could Windows Update (a trusted system process of the operating system), which we all (have no choice but to) trust (and have to do so blindly and completely) have itself installed an untrusted self-signed Lenovo driver (i.e. code running in Kernel-Mode) on a Microsoft Surface device?

Frankly, since this piece of software was signed using a self-signed test cert, who's to say this was even a real Lenovo driver? It could very well be some malicious code purporting to be a Lenovo driver. Or, there is also the remote possibility that it could be a legitimate Lenovo driver, that is self-signed, but if that is the case, its installation should not have been allowed to succeed.



Unacceptable and Deeply Concerning

To us, this is unacceptable, alarming and deeply concerning, and here's why.


We just had, on a device we consider trustworthy (, and could possibly have engaged in business on,) procured from a vendor we consider trustworthy (considering that the entire world's cyber security ultimately depends on them), an unknown, unsigned piece of software of Chinese origin that is now running in Kernel-mode, installed on the device, by this device's vendor's (i.e. Microsoft's) own product (Windows operating system's) update program!

We have not had an opportunity to analyze this code, but if it is indeed malicious in any way, in effect, it would've, unbeknownst to us and for no fault of ours, granted System-level control over a trusted device within our perimeter, to some entity in China.

How much damage could that have caused? Well, suffice it to say that, for they who know Windows Security well, if this was indeed malicious, it would've been sufficient to potentially compromise any organization within which this potentially suspect and malicious package may have been auto-installed by Windows update. (I've elaborated a bit on this below.)

In the simplest scenario, if a company's Domain Admins had been using this device, it would've been Game Over right there!

This leads me to the next question - we can't help but wonder how many such identical Surface devices exist out there today, perhaps at 1000s of organizations, on which this suspicious unsigned Lenovo driver may have been downloaded and installed?

This also leads me to another very important question - Just how much trust can we, the world, impose in Windows Update?

In our case, it just so happened to be, that we happened to be in front of this device during this Windows update process, and that's how we noticed this, and by the way, after it was done, it gave the familiar Your device is upto date message.

Speaking which, here's another equally important question - For all organizations that are using Windows Surface, and may be using it for mission-critical or sensitive purposes (e.g. AD administration), what is the guarantee that this won't happen again?

I ask because if you understand cyber security, then you know, that it ONLY takes ONE instance of ONE malicious piece of software to be installed on a system, to compromise the security of that system, and if that system was a highly-trusted internal system (e.g. that machine's domain computer account had the "Trusted for Unconstrained Delegation" bit set), then this could very likely also aid perpetrators in ultimately gaining complete command and control of the entire IT infrastructure. As I have already alluded to above, if by chance the target/compromised computer was one that was being used by an Active Directory Privileged User, then, it would be tantamount to Game Over right then and there!

Think about it - this could have happened at any organization, from say the U.S. Government to the British Government, or from say a Goldman Sachs to a Palantir, or say from a stock-exchange to an airline, or say at a clandestine national security agency to say at a nuclear reactor, or even Microsoft itself. In short, for absolutely no fault of theirs, an organization could potentially have been breached by a likely malicious piece of software that the operating system's own update utility had downloaded and installed on the System, and in 99% of situations, because hardly anyone checks what gets installed by Windows Update (now that we have to download and install a whopping 600MB patch every Tuesday), this would likely have gone unnoticed!

Again, to be perfectly clear, I'm not saying that a provably malicious piece of software was in fact downloaded and installed on a Microsoft Surface device by Windows Update. What I'm saying is that a highly suspicious piece of software, one that was built and intended to run in Kernel-mode and yet was merely signed with a test certificate, somehow was automatically downloaded and installed on a Microsoft Surface device, and that to us is deeply concerning, because in essence, if this could happen, then even at organizations that may be spending millions on cyber security, a single such piece of software quietly making its way in through such a trusted channel, could possibly instantly render their entire multi-million dollar cyber security apparatus useless, and jeopardize the security of the entire organization, and this could happen at thousands of organizations worldwide.

With full respect to Microsoft and Mr. Nadella, this is deeply concerning and unacceptable, and I'd like some assurance, as I'm sure would 1000s of other CEOs and CISOs, that this will never happen again, on any Surface device, in any organization.

In our case, this was very important, because had we put that brand new Surface device that we procured from none other than the Microsoft Store, into operation (even it we had re-imaged it with an ultra-secure locked-down internal image), from minute one, post the initial Windows update, we would likely have had a potentially compromised device running within our internal network, and it could perhaps have led to us being breached.



If I Were Microsoft, I'd Send a Plane

Dear Microsoft, we immediately quarantined that Microsoft Surface device, and we have it in our possession.


If I were you, I'd send a plane to get it picked up ASAP, so you can thoroughly investigate every little aspect of this to figure out how this possibly happened, and get to the bottom of it! (Petty process note: The Microsoft Store let us keep the device for a bit longer, but will not let us return the device past June 24, and the only reason we've kept it, is in case you'd want to analyze it.)

Here's why. At the very least, if I were still at Microsoft, and in charge of Cyber Security -
  1. I'd want to know how an untrusted Kernel-mode device driver made it into the Windows Catalog
  2. I'd want to know why a Microsoft Surface device downloaded a purportedly Lenovo driver
  3. I'd want to know how Windows 10 permitted and in fact itself installed an untrusted driver
  4. I'd want to know exactly which SKUs of Microsoft Surface this may have happened on
  5. I'd want to know exactly how many such Microsoft Surface devices out there may have downloaded this package 

Further, and as such, considering that Microsoft Corp itself may easily have thousands of Surface devices being used within Microsoft itself, if I were still with Microsoft CorpSec, I'd certainly want to know how many of their own Surface devices may have automatically downloaded and installed this highly suspicious piece of untrusted self-signed software.


In short, Microsoft, if you care as deeply about cyber security as you say you do, and by that I'm referring to what Mr. Nadella, the CEO of Microsoft, recently said (see video below: 0:40 - 0:44) and I quote "we spend over a billion dollars of R&D each year, in building security into our mainstream products", then you'll want to get to the bottom of this, because other than the Cloud, what else could be a more mainstream product for Microsoft today than, Microsoft Windows and Microsoft Surface ?! -



Also, speaking of Microsoft's ecosystem, it indeed is time to help safeguard Microsoft's global ecosystem. (But I digress,)



In Conclusion

Folks, the only reason I decided to publicly share this is because I care deeply about cyber security, and I believe that this could potentially have impacted the foundational cyber security of any, and potentially, of thousands of organizations worldwide.


Hopefully, as you'll agree, a trusted component (i.e. Windows Update) of an operating system that virtually the whole world will soon be running on (i.e. Windows 10), should not be downloading and installing a piece of software that runs in Kernel-mode, when that piece of software isn't even digitally signed by a valid digital certificate, because if that piece of software happened to be malicious, then in doing so, it could likely, automatically, and for no fault of its users, instantly compromise the cyber security of possibly thousands of organizations worldwide. This is really as simple, as fundamental and as concerning, as that. 

All in all, the Microsoft Surface is an incredible device, and because, like Apple's computers, the entire hardware and software is in control of a single vendor, Microsoft has a huge opportunity to deliver a trustworthy computing device to the world, and we'd love to embrace it. Thus, it is vital for Microsoft to ensure that its other components (e.g. Update) do not let the security of its mainstream products down, because per the Principle of Weakest Link, "a system is only as secure as is its weakest link."


By the way, I happen to be former Microsoft Program Manager for Active Directory Security, and I care deeply for Microsoft.

For those may not know what Active Directory Security is (i.e. most CEOs, a few CISOs, and most employees and citizens,) suffice it to say that global security may depend on Active Directory Security, and thus may be a matter of paramount defenses.

Most respectfully,
Sanjay


PS: Full Disclosure: I had also immediately brought this matter to the attention of the Microsoft Store. They escalated it to Tier-3 support (based out of New Delhi, India), who then asked me to use the Windows Feedback utility to share the relevant evidence with Microsoft, which I immediately and dutifully did, but/and I never heard back from anyone at Microsoft in this regard again.

PS2: Another small request to Microsoft - Dear Microsoft, while at it, could you please also educate your global customer base about the paramount importance of Active Directory Effective Permissions, which is the ONE capability without which not a single object in any Active Directory deployment can be adequately secured! Considering that Active Directory is the foundation of cyber security of over 85% of all organizations worldwide, this is important. Over the last few years, we've had almost 10,000 organizations from 150+ countries knock at our doors, and virtually none of them seem to know this most basic and cardinal fact of Windows Security. I couldn't begin to tell you how shocking it is for us to learn that most Domain Admins and many CISOs out there don't have a clue. Can you imagine just how insecure and vulnerable an organization whose Domain Admins don't even know what Active Directory Effective Permissions are, let alone possessing this paramount capability, could be today?