ALC - Ascera

Scroll down to watch!

Tired of dealing with spreadsheet chaos, manual tracking, and disjointed tools as you prepare for CMMC?

In this live conversation, Eric Skiff,...

Across industries, compliance demands are mounting. Whether it’s CMMC, HIPAA, SOX, or ISO 27001, organizations must not only achieve compliance but stay compliant over time.   

Although this journey can be difficult, AI offers a solution. Properly leveraging AI tools can replace tedious documentation and evidence wrangling with continuous monitoring and proactive audit readiness. 

AI is, however, a double-edged sword. If used without guardrails, it can mislead and introduce new risks. This article shows how to strike the balance between harnessing AI’s power while maintaining control, traceability, and trust. 

The Risks of Using AI for Cyber Compliance 

AI offers several advantages for organizations working toward cyber compliance, but it also poses several risks. When used incorrectly — especially if not grounded (read more about grounding in this blog) — AI can output information that’s flat-out wrong.

Most general-purpose AI models are not trained specifically on compliance frameworks. That means they may:

• “Hallucinate” information that sounds confident but is wrong
• Pull outdated or irrelevant advice from internet sources
• Miss critical nuances in control language, leading to incomplete or incorrect implementation

Here are some examples of what this might look like: 

• AI could suggest that for CMMC control CA.L2-3.12.3, all that’s required to perform “continuous compliance monitoring” is to do weekly vulnerability scans and stay within SLA policies for remediation when the continuous monitoring requirement is to “monitor security controls on an ongoing basis to ensure the continued effectiveness of the controls.” 
• An LLM could state there are 17 requirements in CMMC L1, when there are actually only 15 due to a change in the final ruling that understandably would confuse an LLM as it changed very recently. 
• LLMs can give inaccurate guidance on which controls can or cannot be POA&M’d, which is very nuanced in CMMC.  For example, you’re likely to get a wrong answer to: “Can any CMMC controls with a weight of 5 have a POA&M?”  While most will tell you that none can have a POA&M, using an LLM built for CMMC like ComplyAI will give you a more nuanced and up-to-date answer such as the following:  

The Benefits of Using AI for Cyber Compliance 

When used properly, AI can transform the cyber compliance process and eliminate several common pain points including: 

• Manual document creation and updating
• Confusion surrounding ambiguous or weighty regulatory language 
• Burdensome evidence collection and review 
• Gaps emerging just before audits 

While AI doesn’t replace the need for rigorous controls or compliance staff, it can accelerate many tasks and enable a compliance team to scale when used as an assistant. Here are some examples of what AI can bring to the table: 

• Automate drafting of policies, procedures, and evidence templates 
• Interpret ambiguous regulatory or control language 
• Ingest your existing artifacts (policies, controls, assessments) and make them explorable via chat 
• Review evidence and flag mismatches or gaps 
• Run “pre‑audit scans” to detect what assessors might flag

But to do these things safely, you need solid guardrails. That’s where safe AI principles come in. 

Safe AI Principles: Foundation for Responsible Use 

Using AI for cyber compliance can be extremely beneficial when done correctly. Here are some principles to keep in mind to keep your AI usage safe and effective.

 

Ground in Context 

Don’t feed AI generic prompts. Instead, ingest your own policies, control mappings, environment architecture, prior audits, and regulatory references. 

Use a solution that leverages techniques like CAG and RAG and that automatically leverages the context of your compliance data within a private account to avoid having to copy and paste into a public model (like ChatGPT or Google Gemini) and automatically put your data in context — without sensitive compliance data ever leaving your control. 

Check out this blog for a full breakdown of grounding.

 

Keep Humans in the Loop 

AI should support, but humans should decide. Every output (policy draft, evidence conclusion, gap assessment) must be reviewed by domain experts. 

Protect Data 

Never paste sensitive content into public AI prompts. Use secure, enterprise-grade models or private AI deployments. 

Use Traceable Outputs 

Require AI to cite sources (internal docs, regulatory texts, control references). Traceability is essential for trust and audit defense. 

Ensure Vendor Due Diligence 

Your AI tool provider should adhere to robust security, governance, and ethical AI practices. Evaluate against NIST AI RMF, NIST AI 600‑1, and ISO/IEC 42001. 

AI Across the Compliance Lifecycle

Below is a breakdown of common compliance phases and how AI can assist in each.

Phase	AI Use Cases	What Adds Value	What to Watch Out For
Ingestion / Onboarding	Upload policies, procedures, prior assessments, control mappings	Auto-populate fields, build explorable ‘chat your documents’ UI	Garbage in, garbage out
Interpretation	Ask AI to clarify vague regulatory phrasing	Save time by standardizing interpretations	AI may offer inconsistent interpretations
Documentation & Drafting	Generate or update policies, procedures, plans	Reduce authoring effort	Validate tone, coverage, and alignment
Evidence Review & Assessment	Scan evidence and judge compliance	Flag weak or missing evidence	AI may misinterpret context
Audit Prep & Gap Detection	Perform pre-audit scans	Highlight gaps likely flagged	AI may over-predict

Risks, Threats & Mitigations 

Risk	Threat	Mitigation
Data Leakage	Sensitive info uploaded to public models	Use enterprise AI; enforce prompt gating policy
Hallucinated or Outdated Outputs	AI cites nonexistent regulation or outdated control	Require citations; lock to known source corpus
Overconfidence / Blind Trust	Teams accept wrong outputs	Mandate human approval
Weak Vendor Governance	Supplier introduces poor update practices	Conduct vendor audits against NIST/ISO
Audit Surprises	AI misses gaps	Use AI as support, not replacement for compliance teams

 

Final Thoughts 

AI doesn’t replace compliance professionals, but it can enhance them. Used as a collaborator, AI can lift burdensome tasks, reduce cycle times, and help you maintain a living compliance program rather than a stagnant “one-and-done” exercise.

But misuse is dangerous. Misstatements may lead to audit failures or fines. The difference between success and failure comes down to governance, human oversight, traceability, and vendor discipline. 

ASCERA’s ComplyAI AI assistant was built with these safe AI practices in mind. Unlike other AI tools, ComplyAI grounds your data in the context of your environment and never sends data outside of the platform. Our AI assistant was trained specifically on assessor-vetted compliance materials, so you can trust that its outputs are accurate and tailored to your unique security environment. 

Get started with a demo today to witness ComplyAI in action. 

 

When building AI tools for compliance or security work, you might quickly run into a problem: copy-paste fatigue.

LLMs can help users with internal processes, compliance tasks, and questions about policies, but there’s friction. Users first have to first pull data from their tools, copy it over to the LLM, and then complete their task. 

If you do this once or twice, it works well enough. When you’re doing it every day, however, that friction adds up. Users either stop updating the data they’re feeding into the LLM, or they stop using the tool altogether. The result is stale data, hallucinated answers that aren’t anchored to the truth, and no real efficiency gains. 

The solution is to build AI tools that stay grounded in your real environment. Instead of forcing users to copy and paste data, the system already knows the current state. It pulls live information, applies organizational context, and keeps responses anchored to what is true and relevant right now. 

With that foundation, a user can just ask a question or start the task at hand. The system retrieves the relevant policies, parameters, and live operational data, then returns an answer tailored to their environment.

The rest of this article breaks down the architecture required to make that possible. The diagram below provides a glimpse at the complete architecture this article will explore. 

Looking for a higher-level overview of using AI for cyber compliance including safe practices, risks, benefits, and use cases? Read this blog.

 

 

What is RAG? 

 

RAG stands for Retrieval Augmented Generation. It’s the primary technique used to reduce hallucinations and ground LLMs in real data. 

Here’s how it works at a high level: 

1. Take your documents and break them into chunks 
2. Send those chunks through an embedding model, which creates a mathematical representation called a vector or embedding 
3. Store these embeddings in a vector database 
4. When a user asks a question, convert that question into an embedding using the same embedding model 
5. Perform a similarity search to find which embeddings in your database are closely related to the question 
6. Return those embeddings as context to the LLM 

Think of embeddings like flashcards. Each chunk of your document becomes a flashcard, and when you ask a question, the system finds the flashcards that are most relevant to that question and shows them to the LLM. The LLM uses that context to generate a more accurate and grounded response. 

It’s important to note that chunking methods and similarity search techniques are just simple examples here. There are many more sophisticated techniques you can implement to improve this process, including semantic chunking, hierarchical retrieval, hybrid search, and reranking strategies. We won’t be covering these topics in this article, but they are great topics to explore as you refine your RAG pipeline. 

 

Manual RAG vs. Automatic RAG 

There are two approaches to building your RAG pipeline, depending on your data and use case. 

 

Manual RAG 

 

Use manual RAG when you have a smaller set of somewhat structured data, and, most importantly, the quality of how it’s chunked is critical. 

For example, consider a large policy document that’s well-structured into sections. You want to ensure that each chunk contains a complete thought or section. Rather than using an automatic chunking method that might break up sentences or paragraphs arbitrarily, you manually review and chunk the document. You may use a systematic approach to chunk the document, but the human is in the loop to ensure the quality of the chunks. 

 

Automatic RAG 

 

Use automatic RAG when you have a large volume of data with varying structures, and you need to ensure all that data is available for search. 

For example, if you have a document store full of policy documents that are constantly being updated, you can establish a chunking method that works across all of them. You might chunk at every paragraph, or after every 100 words with some overlap between chunks. 

The point is that you don’t need someone to manually review every document and re-chunk it every time it updates. You establish a chunking method that works well, automate it, and let it run. There are several chunking techniques you can use depending on what works best for your documents. 

 

Hybrid Approach 

 

In practice, many organizations use a hybrid approach, applying manual RAG to critical policy documents where precision matters and automatic RAG to larger document repositories where coverage is more important than perfect chunking. 

 

Contextualization 

 

In this step, we ensure that the chunks are contextualized within the document they are part of. This helps to ensure chunks are not isolated pieces of text but are understood in relation to the whole document. 

For instance, you may have a chunk detailing information about a retention policy without explicitly mentioning that it relates to your SIEM. Here is some information you could add using the help of an LLM to ensure the chunk is well contextualized within the document: 

• The year the document was published 
• The name of the document 
• What section the chunk appeared in 

By adding contextual information about the original document, you help the similarity search process return more accurate results and also help the LLM understand the relevance of that chunk. 

 

Adding Metadata 

 

Similar to contextualization, adding metadata to your chunks helps manage them in a more systematic way. Here are some examples of metadata you might add: 

• Page number the chunk came from 
• Filename 
• Hash of the document 

Some of this metadata is useful for auditability and traceability, while other metadata helps manage the lifecycle of your embeddings. For example, storing a hash of the document allows you to detect when a document has changed. You can then use that information to update or archive embeddings as needed. 

 

Embedding 

 

At this stage, the original documents have been chunked, contextualized, and enriched with metadata. Now it’s time to convert those chunks into embeddings. 

An embedding model is used to transform each chunk of text into a vector representation. The embedding model processes the text and generates a high-dimensional vector that captures the semantic meaning of the text. This vector is called an embedding. An embedding may look something like this: 

[0.11, -0.35, 0.13, 0.41…, 0.71]

In this high-dimensional space, embeddings that are semantically similar will be located closer together. This allows us to perform similarity searches later on. 

 

CAG, RAG, and MCP 

 

RAG is just one piece of the puzzle. There are several other tools and techniques you can use to further enhance the LLM’s ability to provide accurate and relevant responses. Here are some additions that can be used at inference time by the LLM to enhance its capabilities and reduce hallucinations: 

 

CAG (Cache/Context Augmented Generation) 

 

CAG provides general context about your organization that gets injected into every conversation automatically. This might include information such as: 

• Your company name 
• The industry you operate in 
• The tools you use (SIEM, vulnerability scanner, etc.) 
• Your compliance frameworks 

Users don’t have to repeat this context every time. It’s present in the background, helping anchor the LLM’s responses to your specific environment. 

 

RAG (Retrieval Augmented Generation) – Vector Based 

 

We’ve already covered RAG in detail, but to reiterate, RAG serves as your knowledge database. Now that we have our knowledge base built into a vector database, here’s a simple implementation of how RAG works at inference time: 

1. The user asks a question 
2. The question is converted into an embedding using the same embedding model 
3. The embedding of the question is used to search the vector database for similar embeddings 
4. The identified embeddings are returned as context to the LLM 

MCP (Model Context Protocol) 

 

MCP is a common protocol that expands the LLM’s capabilities by giving it access to tools. However, MCP isn’t the only way to achieve this. It can be any form of function calling or tool calling. MCP is just one implementation. 

The key concept is that you’re giving the LLM the ability to fetch live data on its own. For example: 

• Query your vulnerability scanner for current vulnerabilities 
• Check your SIEM for recent alerts 
• Pull data from your ticketing system 

Depending on the capability of the tool being queried, the LLM might even construct a custom query. For instance, after RAG provides the SLA ranges, the LLM could build a query to look for tickets that fall outside of those ranges. 

This is how you integrate real-time data into the LLM’s responses without having to manually provide it. 

 

Putting It All Together: A Real Example 

 

Let’s walk through a real query to see how all three tools work together. 

User asks: “Are we maintaining our vulnerability response SLAs?” 

This is a question you can’t simply ask a public LLM without first uploading substantial context. But with a properly built RAG pipeline and integrations, the LLM can answer this question accurately and in real-time. 

Step 1: CAG provides background context 

The LLM already knows from CAG that: 

• Your company is ACME Corp. 

• You use a specific vulnerability scanner 

• You operate in a regulated industry with compliance requirements 

Step 2: RAG retrieves relevant knowledge 

The system converts your question into an embedding and searches the vector database. It might return embeddings such as: 

• A chunk from your vulnerability management policy detailing how SLAs are defined and measured 

• A check on your escalation process when SLAs are breached 

All of this gets added as context to the LLM automatically. 

Step 3: MCP fetches live data 

Now the LLM knows what your SLAs are from RAG, but it needs current vulnerability data. So it makes a tool call to your vulnerability scanner: 

get_vulnerability_details()
 

This returns live data about all your current vulnerabilities, including their severity and how long they’ve been open. The LLM might even construct a more advanced query based on the SLA ranges it learned from RAG, specifically looking for vulnerabilities that exceed those thresholds. 

Step 4: LLM generates the response 

Now the LLM has everything it needs: 

• Company context (CAG) 

• SLA definitions (RAG) 

• Current vulnerability data (MCP) 

It can generate a response such as: 

“ACME Corp. is currently missing SLAs on 3 vulnerabilities that reach their deadlines today: 1 critical vulnerability breaching the 24-hour SLA, 1 high severity vulnerability breaching the 7-day SLA, and 1 medium severity vulnerability breaching the 30-day SLA, all requiring immediate remediation.” 

 

Managing Embedding Lifecycle 

 

One challenge with RAG is keeping your knowledge base current. Here’s how to handle it: 

When a document changes: 

1. Detect the change (via file hash or monitoring your document store) 
2. Query all embeddings that came from the old document 
3. Set their status to “archived” to invalidate them 
4. Create new chunks from the updated document 
5. Generate new embeddings and add them to the database 

This approach ensures that normal queries only return current embeddings. The responses will be based on the latest information. 

If a query requires historical context (e.g., “When did our vulnerability SLAs change?”), you can include archived embeddings in the search to provide the historical perspective. 

Additional logic may be needed to remove outdated embeddings after a certain period of time or other lifecycle management strategies. 

 

Common Pitfalls to Avoid 

 

When building a grounded AI, there are several common mistakes that can undermine your implementation: 

Chunking documents incorrectly: Chunking too small loses context, while chunking too large reduces retrieval precision. Test different chunk sizes with real queries to find the right balance for your documents. 

Not versioning your embeddings: Without proper versioning and archiving, you lose the ability to track how policies and procedures have evolved. Always maintain document hashes and version metadata. 

Over-relying on RAG without human review: AI should assist, not replace human judgment. Implement review workflows for critical outputs, especially for compliance-related decisions. 

Not testing with real user queries: Build your system around actual questions your users ask, not hypothetical scenarios. Collect and analyze real queries to improve your chunking and retrieval strategies. 

Failing to run evaluations after changes: Every time you modify your chunking strategy, update embeddings, or change models, run a comprehensive evaluation suite. Without evals, you won’t catch regressions in quality. 

Not testing different models: Different embedding models and LLMs perform differently on your specific use case. Test multiple options and measure their performance on your actual data. 

Providing too much context to the LLM: More context isn’t always better. Limit the LLM’s context to what it truly needs for each query. Excess context can dilute relevance and increase latency and costs. 

Not making the implementation auditable: For compliance and security work, you need to be able to trace every response back to its sources. Implement logging, citation tracking, and audit trails that show which embeddings and tools were used for each response. 

 

Why This Matters: Reducing Friction 

 

At the end of the day, if it’s more difficult to use your LLM-based tool than it is to retrieve the data manually, then the LLM isn’t providing any value. 

That’s why grounding your AI in context is so important. It’s not just about reducing hallucinations; it’s about building tools that users actually want to use. Tools that add value by reducing the time it takes to retrieve data and make connections. Tools that are as frictionless as possible. 

The goal is to make it easier and faster to use the tool than to perform these tasks manually. When you eliminate the copy-paste fatigue, keep data current automatically, and put information in the hands of users who might not be able to retrieve it themselves, that’s when AI becomes truly valuable for compliance and security work. 

 

ASCERA: Purpose-Built for Compliance and Security 

 

If you’ve made it this far, thank you! Also, you’re probably wondering: “This sounds great, but it sounds like a lot of work to build.” That’s a fair point. 

Building a grounded AI such as ComplyAI with proper integrations, contextualization, and tool calling capabilities is time-consuming. It requires deep expertise in both the technical implementation and the compliance and security domains. 

This is what ASCERA is designed to solve. If you’re a compliance or security professional looking to leverage AI in your work, but you don’t have the time or resources to build this from scratch, you’re not alone. 

ASCERA is purpose-built for compliance and security professionals. We’ve already done the heavy lifting of building out integrations with the tools you use, establishing comprehensive knowledge bases of compliance frameworks, and creating the contextualization layers that make AI responses accurate and actionable. 

Instead of spending months researching integrations and building RAG pipelines, ASCERA provides you with a solution that already understands your environment. It has context about your compliance requirements, security policies, and operational tools built in. 

ASCERA helps organizations not only achieve compliance but maintain continuous compliance through automated processes, real-time control monitoring, and actionable insights. By leveraging ASCERA’s pre-built capabilities, you can focus on what matters: improving your security posture and meeting your compliance objectives. 

As the Cybersecurity Maturity Model Certification (CMMC) framework evolves, defense contractors must ensure they meet strict compliance requirements to handle...

Preparing for compliance audits, especially under the rigorous requirements of the Cybersecurity Maturity Model Certification (CMMC), can be exhausting.Audit fatigue