Module 1: NLP Foundations

What Computers Do with Language

Much of the world's useful information is written in words, not numbers: customer complaints, product reviews, emails, contracts, social media posts and support chats. Natural language processing (NLP) is the set of techniques that let computers work with this text: sorting it, searching it, extracting facts from it and generating it. Understanding what computers actually do with language, and what they do not, helps you choose sensible methods and set realistic expectations.

Computers work with numbers, not meaning

A computer does not read a sentence the way you do. Underneath, text is stored as a sequence of character codes. To analyse text, every NLP method must turn words into numbers in some way. The history of NLP is largely a history of better ways to do that conversion:

• Rules and dictionaries. Hand-written patterns, such as "if a message contains 'refund' and 'not received', route it to the payments team". Simple, transparent, but brittle.
• Counting words. Represent each document by how often each word appears (bag of words, TF-IDF). Surprisingly effective for sorting documents.
• Word embeddings. Represent each word as a list of numbers learned from large amounts of text, so that words used in similar contexts get similar numbers.
• Transformer models. Large neural networks that represent each word in the context of the whole sentence. They power modern translation, search and text generation.

Common NLP tasks

Why language is hard

Language is full of features that are easy for people and difficult for machines:

• Ambiguity. "The bank was closed" could mean a financial institution or a river bank. Context decides.
• Sarcasm and tone. "Great, my order is late again" contains a positive word but expresses frustration.
• Variation. Spelling mistakes, abbreviations and slang, such as "pls", "thx" or "asap".
• Code-switching. In East Africa, many messages mix English, Kiswahili and Sheng in one sentence: "Nimetuma pesa but haijafika". Many tools are trained mostly on English and perform less well on mixed or local-language text.
• Negation. "Not bad at all" is positive, even though it contains "not" and "bad".

Worked example: sorting customer messages

A hypothetical online electronics shop in Nairobi receives about 800 customer messages a week through WhatsApp and email. Staff read every message and forward it to the right team, which delays responses.

A first, rule-based approach might look like this:

def route(message):
    text = message.lower()
    if "refund" in text or "money back" in text or "pesa" in text:
        return "payments"
    if "deliver" in text or "rider" in text or "haijafika" in text:
        return "delivery"
    if "broken" in text or "not working" in text or "warranty" in text:
        return "after-sales"
    return "general"

print(route("Nimetuma pesa but order haijafika"))   # payments

The message is really about a missing delivery after payment, but the rule picks "payments" because that rule comes first. Rules break easily as messages vary. In this course you will learn to replace them with models that learn from labelled examples, measure how well they work and improve them.

What NLP cannot do

Even the best models do not understand text the way people do. They learn statistical patterns from the data they were trained on. They can be confidently wrong, reflect biases in their training data and struggle with text that differs from what they saw before. That is why careful evaluation, which you will learn later in the course, matters so much.

Common mistakes to avoid

• Assuming a tool that works well on English news will work equally well on Sheng customer chats.
• Starting with the most complex model before trying a simple baseline.
• Ignoring the privacy of text data: messages often contain names, phone numbers and account details.
• Expecting a model to "understand" rather than measuring what it actually gets right.