Introduction
This is the seventh session of the Expert System course, and we’re gonna implement a Lexical Analyzer class inside our project.
A Lexical Analyzer detects characteristics of words from a formatted source. Imagine the compiler tasks to detect keywords and valid typing.
#include <QStack>
How does the compiler knows that we’re typing a header?
QFile* file=m_inputs.top();
m_inputs.pop()->close(); delete file;
How does the compiler knows that delete is a keyword?
This is the work of the Lexical Analyzer in every Compiler.
This is helpful because, we’re able to introduce a file with a certain format and take every rule from it, instead of creating rules manually (as we did in the last video).
Code
First, we should create a new class named PROPLexicalAnalyzer, and add a few headers:
#include <QStack> #include <QChar> #include <QString> #include <QFile> #include <QTextStream>
The QStack class is a container that allows us push and pop element only from one site, imagine a book stack,where you are able to take or leave books only from the top.
A QTextStream is a writing and reading stream for QStrings ( that are sequence of Qchars), similar than C++ strings. QTextStream let us join all C++ standard stream in only one class.
class PROPLexAnalyzer { protected: QStack<QFile*> m_inputs; QStack<QChar> m_pendingChars; QTextStream m_Stream; int m_LastPos;
The LEXEM struct defines the type of every word, gives meaning defining a specific enumeration of types and links this type to the specific word. It’s worth to keep a index to the tree to every LEXEM
public: //Embedded struct to give lexical meaning to words from a file, and be useful //to the tree structure. struct LEXEM{ enum Type{OPERATOR,ID,ENDOFF,ERROR,OPENBRACKET,CLOSEDBRAKETS,EOL,COMMENT}; Type m_type; QString m_sToken; int m_ntreeIndex; }; LEXEM buildToken(LEXEM::Type,QString token); bool pushFile(QString fileName); void popFile(); QString read(); LEXEM getToken(); PROPLexAnalyzer(); ~PROPLexAnalyzer(); }; #endif // PROPLEXANALYZER_H
buildToken() let us create LEXEMs manually, mainly for testing purpose or external insertion of lexems. PushFile() allows us to add a file to the stack of files and the popFile() eliminates a file from the stack of files. read() gives us the opportunity to store all the file into a QString, and , getToken() separate and determine the type,token and index of a LEXEM from the file.
The next amazing video explains this article while implementing it:
So that’s it for this video, I hope you enjoy it and I remind you to leave a comment if you have any question . Thank you.
References
Architecture: Pedro Arturo Cornejo Torres
C++/Qt Developer: Carlos Enrique Hernández Ibarra