Introduction

This is the seventh session of the Expert System course, and we’re gonna implement a Lexical Analyzer class inside our project.

A Lexical Analyzer detects characteristics of words from a formatted source. Imagine the compiler tasks to detect keywords and valid typing.

#include <QStack>

How does the compiler knows that we’re typing a header?

QFile* file=m_inputs.top();

m_inputs.pop()->close();
delete file;

How does the compiler knows that delete is a keyword?

This is the work of the Lexical Analyzer in every Compiler.

This is helpful because, we’re able to introduce a file with a certain format and take every rule from it, instead of creating rules manually (as we did in the last video).

Code

First, we should create a new class named PROPLexicalAnalyzer, and add a few headers:

#include <QStack>
#include <QChar>
#include <QString>
#include <QFile>
#include <QTextStream>

The QStack class is a container that allows us push and pop element only from one site, imagine a book stack,where you are able to take or leave books only from the top.

A QTextStream is a writing and reading stream for QStrings ( that are sequence of Qchars), similar than C++ strings. QTextStream let us join all C++ standard stream in only one class.

 

class PROPLexAnalyzer
{
protected:
    QStack<QFile*> m_inputs;
    QStack<QChar> m_pendingChars;
    QTextStream m_Stream;
    int m_LastPos;

The LEXEM struct defines the type of every word, gives meaning defining a specific enumeration of types and links this type to the specific word. It’s worth to keep a index to the tree to every LEXEM

public:
    //Embedded struct to give lexical meaning to words from a file, and be useful
    //to the tree structure.
    struct LEXEM{
        enum Type{OPERATOR,ID,ENDOFF,ERROR,OPENBRACKET,CLOSEDBRAKETS,EOL,COMMENT};
        Type m_type;
        QString m_sToken;
        int m_ntreeIndex;
    };

    LEXEM buildToken(LEXEM::Type,QString token);
    bool pushFile(QString fileName);
    void popFile();
    QString read();
    LEXEM getToken();
    PROPLexAnalyzer();
    ~PROPLexAnalyzer();
};
#endif // PROPLEXANALYZER_H

buildToken() let us create LEXEMs manually, mainly for testing purpose or external insertion of lexems. PushFile() allows us to add a file to the stack of files and the popFile() eliminates a file from the stack of files. read() gives us the opportunity to store all the file into a QString, and , getToken() separate and determine the type,token and index of a LEXEM from the file.

The next amazing video explains this article while implementing it:

 

References

Qt Documentation

Architecture: Pedro Arturo Cornejo Torres
C++/Qt Developer: Carlos Enrique Hernández Ibarra