Hive Setup is quite straightforward could be achieved by following just 1,2,3 steps. Below is brief description how to setup Hive. Continue reading
Hadoop’s org.apache.hadoop.fs.FileSystem is generic class to access and manage HDFS files/directories located in distributed environment. File’s content stored inside datanode with multiple equal large sizes of blocks (e.g. 64 MB), and namenode keep the information of those blocks and Meta information. FileSystem read and stream by accessing blocks in sequence order. FileSystem first get blocks information from NameNode then open, read and close one by one. It opens first blocks once it complete then close and open next block. HDFS replicate the block to give higher reliability and scalability and if client is one of the datanode then it tries to access block locally if fail then move to other cluster datanode. Continue reading
HDFS stores file in multiple equal large size block e.g. 64 MB, 128 MB etc. and MapReduce framework access and process these files in distributed environment.
The MapReduce framework works on key-value pairs, it has two key part Mapper and Reducer.Map Reducers read file and split and pass to Mapper. Mapper set the input as key-value pairs and pass to the intermediate for sorting and shuffling. Reducer takes the key and list of value, process and writes to the disk. Continue reading
Hadoop 2.x is based on YARN architecture, which uses ResourceManagaer and ApplicationManager. ResourceManagaer manage recourses across cluster and Application Manager manages job life cycles. Continue reading
Grails is an Open Source, full stack, web application framework for the JVM. It takes advantage of the Groovy programming language and convention over configuration to provide a productive and stream-lined development experience. Below diagram depicts the high level Grails Architecture:
GrailsUnitTestCase and GroovyTestCase both extend JUnit 3.x TestCase.MVC Unit Test case will extend GrailsUNitTest case and delegate the test case control to ControllerUnitTestCase and TagLibUnitTestCase
Grails Service uses to separate your business logic from the other layer or component. Services are named in the format SomeNameService.groovy and placed in the directory /grails-app/services/. Services can make use of dependency injection features, and you can easily call these services from within controllers. Continue reading