Unit 1: Introduction [6 Hours]
This unit builds the foundation for the whole subject. It clarifies how raw data turns into usable knowledge, introduces the two disciplines built around that knowledge (engineering and management), shows what AI brings to the table, and ends with the system that ties it all together: the Knowledge-Based System. Every later unit assumes you're solid on these four ideas.
1.1 Overview of Data, Information and Knowledge
Core Definition
Data is the collection of raw, unprocessed facts, figures, or symbols that carry no meaning by themselves. Information is data that has been organized, structured, or processed to give it context and meaning. Knowledge is information that has been understood, connected with experience and judgment, and made usable for decision-making and action.
Picture a kitchen. Vegetables sitting on the counter are data: real, present, but useless until someone does something with them. A recipe that arranges those vegetables into a method is information: it now has structure and purpose. Knowing, from years of cooking, exactly when to pull the pan off the heat because the smell changed, that's knowledge: experience layered on top of information.
Detailed Explanation
These three form a hierarchy usually called the DIKW hierarchy (Data → Information → Knowledge → Wisdom), and every stage is a transformation of the one before it.
Data → Information: this happens through processing: organizing, summarizing, classifying, or putting data into context. A single number means nothing on its own; grouped and labeled correctly, it starts to say something.
Information → Knowledge: this happens through interpretation: connecting information with prior experience, rules, or expertise so a person (or a system) can reason with it and decide what to do next.
Example from computing: individual sensor readings from a machine (23°C, 24°C, 41°C, 24°C) are data. A report showing "average temperature rose 15°C in the last hour" is information. An engineer concluding "this pattern means the cooling fan is failing, replace it" is knowledge, because it combines the information with prior experience of what such patterns usually mean.
This distinction matters directly for this subject: a plain database stores data and information, but a Knowledge-Based System (covered in 1.4) is built specifically to store and apply knowledge, the reasoning layer on top.
Let's Break It Down
Think of a college result sheet. The marks each student scored are data, just numbers against roll numbers. The class average and pass percentage calculated from those marks is information, since it's now organized and gives a picture. When the subject teacher looks at that average, notices it dropped sharply compared to last year, and concludes "this topic wasn't taught clearly enough, I need to revise my teaching method," that conclusion is knowledge.
| Aspect | Data | Information | Knowledge |
|---|---|---|---|
| Nature | Raw, unorganized | Organized, structured | Understood, applied |
| Context | No context | Has context | Has context + experience |
| Usability | Not directly usable | Usable for understanding a situation | Usable for decision-making and action |
| Example | 41°C, 24°C, 23°C | "Temperature rose 15°C this hour" | "The cooling fan is failing, replace it" |
| Produced by | Sensors, observation, records | Processing, summarizing data | Interpretation, experience, reasoning |
Common Mistake
Students often use "information" and "knowledge" interchangeably. A weather report showing "70% chance of rain today" is information. Deciding to carry an umbrella because you know this app has been reliable in the monsoon season is knowledge. Information tells you the state of things; knowledge tells you what to do about it.
Test Yourself
- In a college attendance system, give one example each of data, information, and knowledge.
- Why is knowledge often described as "information plus experience"?
Answers: 1) Data: daily attendance marks (P/A) per student. Information: "Student X has 65% attendance this semester." Knowledge: "Student X is below the 75% requirement and will be barred from exams unless attendance improves."
2) Because information only describes a situation; it takes prior experience or expertise to interpret what that situation means and to decide the right action, which is what turns it into knowledge.
Summary
- Data is raw and meaningless without processing; information is data with structure and context.
- Knowledge is information combined with experience and judgment, and is directly usable for decisions.
- The DIKW hierarchy (Data → Information → Knowledge → Wisdom) shows the progressive transformation.
- Ordinary databases handle data and information; Knowledge-Based Systems are built to handle knowledge.
One-Liner Revision
Data is raw facts, information is data with context, and knowledge is information made actionable through experience.
1.2 Knowledge Engineering and Knowledge Management
Core Definition
Knowledge Engineering (KE) is the discipline of eliciting, structuring, formalizing, and organizing knowledge from human experts or other sources into a form that a computer system, typically an expert system, can store and reason with. Knowledge Management (KM) is the systematic organizational process of creating, capturing, storing, sharing, and applying knowledge so that people in an organization can find and reuse it effectively.
Knowledge engineering builds smart software by extracting an expert's know-how and feeding it into a machine that can reason like that expert. Knowledge management is how an organization, not a single machine, keeps track of everything it collectively knows so no one has to relearn it from scratch every time.
Detailed Explanation
Both disciplines deal with the "knowledge" layer from 1.1, but they operate at different scales and with different goals.
The Knowledge Engineering process typically follows these steps:
Knowledge identification → Knowledge acquisition/elicitation (interviewing domain experts, studying documents) → Knowledge representation (encoding it as rules, frames, or semantic networks) → Knowledge validation and verification (checking it's correct and consistent) → Inference engine design (building the reasoning mechanism) → Implementation and testing → Maintenance and refinement as the domain changes.
The Knowledge Management cycle typically follows these steps:
Knowledge creation (new insight generated by employees) → Knowledge capture/codification (writing it down, documenting it) → Knowledge storage (in repositories, wikis, databases) → Knowledge sharing/dissemination (across teams) → Knowledge application (using it to solve real problems) → Knowledge review and updating.
The key difference: Knowledge Engineering is a technical, AI-driven discipline aimed at building an intelligent system that automates expert-level reasoning in a narrow domain. Knowledge Management is a broader, organizational and managerial discipline aimed at making sure human knowledge across an entire company doesn't get lost, and it doesn't necessarily involve AI at all, it may just be good documentation practice and the right culture.
| Aspect | Knowledge Engineering | Knowledge Management |
|---|---|---|
| Focus | Building an intelligent, reasoning computer system | Managing an organization's collective knowledge |
| Scope | Narrow, domain-specific (one expert system) | Broad, organization-wide |
| Core tools | AI techniques: rules, inference engines, ontologies | Documentation, wikis, repositories, culture, IT systems |
| Output | A working Knowledge-Based System | Shared, reusable organizational knowledge |
| Practitioner | Knowledge engineer (technical, AI background) | Knowledge/KM manager (often non-technical) |
Common Mistake
Students assume KE and KM are just two names for the same thing because both contain the word "knowledge." KE is specifically about building an AI-based reasoning system from expert knowledge. KM is about how a whole organization handles knowledge as an asset, and doesn't require building any intelligent system at all.
Test Yourself
- List the main steps of the Knowledge Engineering process in order.
- Give an example of a task that falls under Knowledge Management but not Knowledge Engineering.
Answers: 1) Identification → Acquisition/Elicitation → Representation → Validation → Inference engine design → Implementation and testing → Maintenance.
2) A company maintaining an internal wiki where employees document how they solved past client issues, so future employees can search and reuse that knowledge. No AI reasoning system is involved.
Summary
- Knowledge Engineering extracts expert knowledge and encodes it for use by an intelligent system.
- Knowledge Management is the organizational process of creating, storing, sharing, and applying knowledge.
- KE follows a technical process ending in a working system; KM follows a cyclical, ongoing organizational process.
- KE is narrow and AI-driven; KM is broad and can be entirely non-technical.
One-Liner Revision
Knowledge Engineering builds a smart system from expert knowledge; Knowledge Management keeps an organization's collective knowledge usable.
1.3 Artificial Intelligence Use in Knowledge Engineering
Core Definition
Artificial Intelligence (AI) supplies the core techniques that Knowledge Engineering (2.1's KE process) relies on to represent captured knowledge inside a computer and to reason with it automatically, turning a static collection of facts and rules into a system that can actually think through problems the way an expert would.
Knowledge Engineering needs some way to make a computer "reason." AI is where that reasoning machinery comes from: the methods for storing knowledge in a usable shape and the algorithms for drawing conclusions from it.
Detailed Explanation
AI contributes to Knowledge Engineering at almost every stage of the KE process described in 1.2. The major contributions are:
Knowledge representation techniques: production rules (IF-THEN statements), frames, semantic networks, and ontologies are all AI methods used to encode expert knowledge into a structured, machine-readable form during the representation stage of KE.
Inference and reasoning techniques: forward chaining, backward chaining, and fuzzy logic are AI reasoning algorithms that let the system derive new conclusions from the represented knowledge, this is what powers the inference engine stage of KE (explained further in 1.4).
Machine learning: where expert interviews are limited or expensive, ML algorithms can automatically detect patterns and rules from large historical datasets, supplementing manual knowledge elicitation with a data-driven route to acquiring knowledge.
Natural Language Processing (NLP): helps knowledge engineers extract knowledge from unstructured sources, manuals, research papers, or transcripts of expert interviews, speeding up the acquisition stage.
Expert system shells: AI-based software frameworks (such as CLIPS or Prolog-based shells) give knowledge engineers a ready-made inference engine and representation format, so they don't have to build the reasoning machinery from scratch for every project.
| AI technique | Role in Knowledge Engineering |
|---|---|
| Rules, frames, semantic networks, ontologies | Represent expert knowledge in machine-usable form |
| Forward/backward chaining, fuzzy logic | Reason with the represented knowledge to reach conclusions |
| Machine learning | Extract knowledge/patterns automatically from data |
| NLP | Extract knowledge from unstructured text sources |
| Expert system shells | Provide ready-made inference engines to speed up building |
Common Mistake
Many students assume "AI in Knowledge Engineering" means machine learning only. Historically, classical symbolic AI (rules, logic, semantic networks) is the core of KE, since expert knowledge is usually elicited as explicit rules, not learned from data. Machine learning is a newer, complementary route, not a replacement for it.
Test Yourself
- Which AI technique is used specifically during the knowledge acquisition stage when the source is unstructured text?
- Why can't Knowledge Engineering rely on AI reasoning techniques alone, without representation techniques?
Answers: 1) Natural Language Processing (NLP), since it extracts structured knowledge from documents, manuals, or interview transcripts.
2) Reasoning techniques like forward or backward chaining only work on knowledge that has already been encoded in a structured form (rules, frames). Without representation techniques first putting the knowledge into that form, there is nothing for the inference engine to reason over.
Summary
- AI supplies both the representation methods and the reasoning algorithms that Knowledge Engineering depends on.
- Rules, frames, semantic networks, and ontologies represent knowledge; chaining and fuzzy logic reason with it.
- Machine learning and NLP support the acquisition stage, extracting knowledge from data or text respectively.
- Symbolic AI (rules and logic) remains the historical core of KE; ML is a modern complement.
One-Liner Revision
AI gives Knowledge Engineering its representation formats and reasoning algorithms, turning captured expert knowledge into a system that can think.
1.4 Knowledge-Based System and its Applications
Core Definition
A Knowledge-Based System (KBS) is a computer program that stores domain knowledge in a knowledge base and uses an inference engine to reason over it, solving problems, answering queries, or giving advice the way a human expert in that domain would.
It's software that "knows" facts and rules about a specific topic, the way a doctor knows medicine, and can answer questions or make suggestions using that knowledge, instead of just retrieving stored records like a plain database would.
Detailed Explanation
A KBS is the end product that Knowledge Engineering (1.2) builds, using the AI techniques from 1.3, on top of the knowledge captured as described in 1.1. Its architecture has these components:
Knowledge base: stores the domain facts, rules, and relationships that were elicited and represented during the KE process. This is the "what the system knows."
Inference engine: the reasoning component that applies logical rules to the knowledge base and the current facts to derive new conclusions or answer a query. This is the "how the system thinks."
Working memory: holds the facts specific to the current problem or session being solved, kept separate from the permanent knowledge base.
User interface: lets the user enter queries or facts and receive the system's conclusions or advice.
Explanation facility: lets the system justify why or how it reached a conclusion, which builds user trust, especially important since a KBS is meant to replace expert judgment.
Knowledge acquisition module: allows new knowledge to be added or existing knowledge to be updated over time without rebuilding the whole system.
The inference engine reasons in one of two directions:
Forward chaining (data-driven): Known facts → matching rules fire → new facts derived → repeat until a conclusion is reached. It starts from what is known and works toward a conclusion.
Backward chaining (goal-driven): Hypothesis/goal set → system checks which rules could prove it → checks if the required facts exist → confirms or rejects the hypothesis. It starts from a possible conclusion and works backward to see if the evidence supports it.
Let's Break It Down
Think of a doctor diagnosing a patient. Forward chaining is like recording every symptom first, fever, chills, sweating, then letting those symptoms trigger the rule "these three together usually mean malaria." Backward chaining is like the doctor already suspecting malaria from experience, and then specifically checking whether fever, chills, and sweating are present to confirm or rule it out.
| Aspect | Forward chaining | Backward chaining |
|---|---|---|
| Direction | Facts → conclusion | Goal → facts |
| Driven by | Available data | A hypothesis to prove |
| Best suited for | Situations with many possible conclusions | Situations with one specific goal to verify |
| Example use | Monitoring systems, planning | Diagnostic systems, troubleshooting |
Applications of Knowledge-Based Systems:
Medical diagnosis systems (such as MYCIN, which diagnosed bacterial infections), configuration systems (such as XCON, used to configure computer hardware orders), financial and investment advisory systems, fault diagnosis systems in engineering and manufacturing, agricultural advisory systems that guide farmers on crop and pest decisions, tax and legal advisory systems, and technical help-desk and customer support systems.
Common Mistake
Students often think a Knowledge-Based System is just a database with a search feature. A database only stores and retrieves data or information exactly as entered. A KBS stores rules and facts, and its inference engine can derive new conclusions that were never explicitly stored, which is the whole point of the reasoning layer.
Test Yourself
- Name the six main components of a Knowledge-Based System.
- Would a medical diagnosis system that starts by suspecting a specific disease and checks for confirming symptoms use forward or backward chaining? Why?
Answers: 1) Knowledge base, inference engine, working memory, user interface, explanation facility, knowledge acquisition module.
2) Backward chaining, because it starts from a specific hypothesis (the suspected disease) and works backward to check whether the supporting facts (symptoms) are present, rather than starting from all available facts.
Summary
- A KBS stores domain knowledge and reasons over it using an inference engine, unlike a plain database.
- Its core components are the knowledge base, inference engine, working memory, user interface, explanation facility, and knowledge acquisition module.
- Forward chaining reasons from facts to a conclusion; backward chaining reasons from a goal back to supporting facts.
- KBS applications span medicine, configuration, finance, engineering, agriculture, law, and technical support.
One-Liner Revision
A Knowledge-Based System pairs a knowledge base with an inference engine to reason like an expert, using forward or backward chaining, across domains like medicine, engineering, and finance.
Whole Chapter Summary
- 1.1 Data is raw fact, information is organized data, and knowledge is information applied through experience, the DIKW hierarchy underlies everything that follows.
- 1.2 Knowledge Engineering builds a single intelligent system from expert knowledge; Knowledge Management keeps an entire organization's knowledge usable.
- 1.3 AI supplies KE with representation techniques (rules, frames, ontologies) and reasoning techniques (chaining, fuzzy logic, ML, NLP).
- 1.4 A Knowledge-Based System is the working product of KE: a knowledge base plus an inference engine, reasoning via forward or backward chaining, applied across domains like medicine and engineering.
Key Points Sheet
| Concept | Meaning |
|---|---|
| DIKW hierarchy | Data → Information → Knowledge → Wisdom |
| KE process | Identification → Acquisition → Representation → Validation → Inference design → Implementation → Maintenance |
| KM cycle | Creation → Capture → Storage → Sharing → Application → Review |
| Forward chaining | Facts → rules fire → new facts → conclusion (data-driven) |
| Backward chaining | Goal → check supporting rules/facts → confirm/reject (goal-driven) |
Last-Minute Revision Sheet
Data → raw, no meaning. Information → data + context. Knowledge → information + experience, actionable.
KE → builds one AI system from expert knowledge. KM → manages knowledge across a whole organization.
AI in KE → rules/frames/ontologies represent, chaining/fuzzy logic reason, ML/NLP acquire.
KBS → knowledge base + inference engine, not a plain database.
Forward chaining → data-driven, facts to conclusion. Backward chaining → goal-driven, goal to facts.
KBS components → knowledge base, inference engine, working memory, UI, explanation facility, acquisition module.
KBS examples → MYCIN (medical), XCON (configuration).