Summary -
In this topic, we described about the Architecture in detail.
Let’s discuss about each component in detail.
- User Interface (UI)
- Interface to submit queries by the user
- The below are the list of UIs mostly used
- Web based GUI
- Command line interface
- HD Insight
- Driver & compiler
- The Driver component receives the queries
- The compiler component parses the query
- The compiler component creates the execution plan for the query after looking into the table structure and partition metadata from metastore.
- Metastore
- The component stores all the table structure information which includes column information as well.
- Execution engine
- The component executes the execution plan created by the compiler.
- It also manages the different dependencies between the various stages and executes it on appropriate system.
- HDFS or HBase
- Data storage technique to store the data.
- Contains the data which is retrieved by Execution engine to send as results to UI.
Hive interaction with Hadoop -
| Sequence | Operation | Description |
|---|---|---|
| 1 | executeQuery | The Query submitted from User Interface |
| 2 | getPlan | The driver passes the query to compiler to check syntax of the query and query plan of the query |
| 3 | getMetaData | The compiler sends the request to metastore |
| 4 | sendMetaData | Metastore sends the metadata to the compiler |
| 5 | sendPlan | Compiler verifies the requirement and sends the plan to driver |
| 6 | executePlan | Driver send the execution plan to execution engine |
| 7 | metaDataOps for DDLs | Execution engine gets the Metadata DDLs for table data from METASOTRE if required |
| 8 | executeJob | Execution Engine sends the job to JOB TRACKER and JOB Tracker will execute the job |
| 9 | jobDone | JOB TRACKER sends the jobDone Status to EXECUTION ENGINE along with the job output |
| 10 | sendResults | EXECUTION ENGINE sends the results to DRIVER |
| 11 | fetchResults | UI will fetch the results from DRIVER |