Efficient Big Data Processing In Hadoop Mapreduce Pdf Download UPDATED

byHaber Whouses - 0

Efficient Big Data Processing In Hadoop Mapreduce Pdf Download

Parallel programming model

MapReduce is a programming model and an associated implementation for processing and generating large data sets with a parallel, distributed algorithm on a cluster.[1] [2] [3]

A MapReduce plan is equanimous of a map procedure, which performs filtering and sorting (such equally sorting students by first name into queues, one queue for each proper noun), and a reduce method, which performs a summary operation (such equally counting the number of students in each queue, yielding proper noun frequencies). The "MapReduce Arrangement" (also chosen "infrastructure" or "framework") orchestrates the processing by marshalling the distributed servers, running the diverse tasks in parallel, managing all communications and data transfers between the various parts of the system, and providing for redundancy and fault tolerance.

The model is a specialization of the split-utilise-combine strategy for data assay.[4] It is inspired past the map and reduce functions commonly used in functional programming,[5] although their purpose in the MapReduce framework is not the same as in their original forms.[vi] The key contributions of the MapReduce framework are not the actual map and reduce functions (which, for case, resemble the 1995 Message Passing Interface standard's[seven] reduce [eight] and besprinkle [9] operations), only the scalability and fault-tolerance accomplished for a variety of applications by optimizing the execution engine[ citation needed ]. Equally such, a unmarried-threaded implementation of MapReduce is usually not faster than a traditional (non-MapReduce) implementation; any gains are usually only seen with multi-threaded implementations on multi-processor hardware.[x] The use of this model is benign only when the optimized distributed shuffle performance (which reduces network communication cost) and fault tolerance features of the MapReduce framework come up into play. Optimizing the advice price is essential to a good MapReduce algorithm.[11]

MapReduce libraries have been written in many programming languages, with different levels of optimization. A popular open-source implementation that has support for distributed shuffles is part of Apache Hadoop. The name MapReduce originally referred to the proprietary Google engineering, only has since been genericized. By 2014, Google was no longer using MapReduce as their primary large information processing model,[12] and development on Apache Mahout had moved on to more capable and less disk-oriented mechanisms that incorporated total map and reduce capabilities.[13]

Overview [edit]

MapReduce is a framework for processing parallelizable issues across large datasets using a large number of computers (nodes), collectively referred to as a cluster (if all nodes are on the same local network and utilise like hardware) or a filigree (if the nodes are shared across geographically and administratively distributed systems, and use more than heterogeneous hardware). Processing can occur on data stored either in a filesystem (unstructured) or in a database (structured). MapReduce can take advantage of the locality of data, processing it near the identify it is stored in order to minimize communication overhead.

A MapReduce framework (or organisation) is usually composed of three operations (or steps):

  1. Map: each worker node applies the map function to the local data, and writes the output to a temporary storage. A main node ensures that only one re-create of the redundant input data is processed.
  2. Shuffle: worker nodes redistribute data based on the output keys (produced by the map function), such that all data belonging to 1 key is located on the aforementioned worker node.
  3. Reduce: worker nodes now procedure each group of output data, per key, in parallel.

MapReduce allows for the distributed processing of the map and reduction operations. Maps can be performed in parallel, provided that each mapping operation is independent of the others; in practice, this is limited by the number of independent data sources and/or the number of CPUs near each source. Similarly, a set of 'reducers' tin can perform the reduction phase, provided that all outputs of the map operation that share the same key are presented to the same reducer at the aforementioned time, or that the reduction office is associative. While this process frequently appears inefficient compared to algorithms that are more sequential (considering multiple instances of the reduction process must be run), MapReduce can be applied to significantly larger datasets than a unmarried "article" server can handle – a big server farm tin use MapReduce to sort a petabyte of information in simply a few hours.[fourteen] The parallelism also offers some possibility of recovering from fractional failure of servers or storage during the operation: if one mapper or reducer fails, the work can be rescheduled – assuming the input data are still available.

Another way to look at MapReduce is as a five-pace parallel and distributed ciphering:

  1. Prepare the Map() input – the "MapReduce organisation" designates Map processors, assigns the input central K1 that each processor would piece of work on, and provides that processor with all the input data associated with that key.
  2. Run the user-provided Map() code – Map() is run exactly in one case for each K1 central, generating output organized by key K2.
  3. "Shuffle" the Map output to the Reduce processors – the MapReduce system designates Reduce processors, assigns the K2 central each processor should piece of work on, and provides that processor with all the Map-generated information associated with that key.
  4. Run the user-provided Reduce() lawmaking – Reduce() is run exactly once for each K2 central produced past the Map step.
  5. Produce the last output – the MapReduce arrangement collects all the Reduce output, and sorts it by K2 to produce the final result.

These v steps can be logically thought of as running in sequence – each footstep starts only later the previous footstep is completed – although in exercise they can be interleaved equally long every bit the final effect is not affected.

In many situations, the input data might have already been distributed ("sharded") among many different servers, in which case step 1 could sometimes be greatly simplified by assigning Map servers that would process the locally present input information. Similarly, step 3 could sometimes be sped up by assigning Reduce processors that are as close equally possible to the Map-generated information they need to process.

Logical view [edit]

The Map and Reduce functions of MapReduce are both defined with respect to data structured in (cardinal, value) pairs. Map takes 1 pair of data with a type in one data domain, and returns a list of pairs in a dissimilar domain:

Map(k1,v1) → list(k2,v2)

The Map part is applied in parallel to every pair (keyed by k1) in the input dataset. This produces a list of pairs (keyed by k2) for each call. After that, the MapReduce framework collects all pairs with the same primal (k2) from all lists and groups them together, creating one group for each key.

The Reduce function is then applied in parallel to each group, which in turn produces a collection of values in the aforementioned domain:

Reduce(k2, list (v2)) → list((k3, v3)) [15]

Each Reduce call typically produces either 1 key value pair or an empty return, though i call is allowed to return more than than one key value pair. The returns of all calls are collected as the desired event list.

Thus the MapReduce framework transforms a list of (key, value) pairs into another list of (fundamental, value) pairs.[sixteen] This beliefs is different from the typical functional programming map and reduce combination, which accepts a list of arbitrary values and returns ane single value that combines all the values returned by map.

It is necessary merely not sufficient to have implementations of the map and reduce abstractions in order to implement MapReduce. Distributed implementations of MapReduce require a means of connecting the processes performing the Map and Reduce phases. This may exist a distributed file system. Other options are possible, such as direct streaming from mappers to reducers, or for the mapping processors to serve up their results to reducers that query them.

Examples [edit]

The approved MapReduce example counts the appearance of each word in a set of documents:[17] 

          part          map(String name, String document):          // name: document name          // document: certificate contents          for each          word due west          in          document:     emit (w, ane)          function          reduce(String word, Iterator partialCounts):          // word: a word          // partialCounts: a list of aggregated partial counts          sum = 0          for each          pc          in          partialCounts:     sum += pc   emit (word, sum)

Here, each document is split into words, and each word is counted by the map function, using the word as the effect key. The framework puts together all the pairs with the aforementioned cardinal and feeds them to the same phone call to reduce. Thus, this part simply needs to sum all of its input values to find the full appearances of that discussion.

As another instance, imagine that for a database of 1.1 billion people, one would similar to compute the boilerplate number of social contacts a person has according to age. In SQL, such a query could exist expressed as: 

                        SELECT            age            ,            AVG            (            contacts            )            FROM            social            .            person            GROUP            By            age            Order            Past            historic period

Using MapReduce, the K1 fundamental values could be the integers 1 through 1100, each representing a batch of i million records, the K2 cardinal value could be a person'southward historic period in years, and this ciphering could be achieved using the following functions: 

          function          Map          is          input:          integer          K1 between 1 and 1100, representing a batch of i million social.person records          for each          social.person record in the K1 batch          practice          let          Y be the person'south age          let          Northward be the number of contacts the person has          produce one output record          (Y,(Due north,1))          repeat          stop function          function          Reduce          is          input:          age (in years) Y          for each          input tape (Y,(N,C))          do          Accrue in S the sum of N*C         Accumulate in Cnew          the sum of C          repeat          permit          A be South/Cnew          produce one output record          (Y,(A,Cnew))          terminate function

Annotation that in the Reduce function, C is the count of people having in full North contacts, so in the Map role it is natural to write C=1, since every output pair is referring to the contacts of one single person.

The MapReduce system would line upward the 1100 Map processors, and would provide each with its respective 1 1000000 input records. The Map step would produce 1.i billion (Y,(Due north,one)) records, with Y values ranging between, say, viii and 103. The MapReduce System would then line upward the 96 Reduce processors past performing shuffling operation of the primal/value pairs due to the fact that nosotros need average per age, and provide each with its millions of corresponding input records. The Reduce step would result in the much reduced set of only 96 output records (Y,A), which would be put in the concluding issue file, sorted by Y.

The count info in the record is important if the processing is reduced more i time. If we did not add the count of the records, the computed boilerplate would be incorrect, for instance: 

          -- map output #1: historic period, quantity of contacts          10, ix 10, 9 10, 9        
          -- map output #2: age, quantity of contacts          10, 9 10, ix        
          -- map output #three: historic period, quantity of contacts          10, ten

If we reduce files #1 and #2, we will have a new file with an average of ix contacts for a 10-year-old person ((9+9+9+nine+9)/v): 

          -- reduce footstep #i: historic period, average of contacts          ten, 9

If we reduce it with file #three, we lose the count of how many records we've already seen, and so we finish up with an average of 9.5 contacts for a 10-twelvemonth-old person ((9+x)/two), which is wrong. The right answer is nine.166 = 55 / vi = (9*3+9*2+10*i)/(3+2+1).

Dataflow [edit]

Software framework architecture adheres to open-closed principle where code is finer divided into unmodifiable frozen spots and extensible hot spots. The frozen spot of the MapReduce framework is a large distributed sort. The hot spots, which the application defines, are:

  • an input reader
  • a Map function
  • a sectionalization office
  • a compare function
  • a Reduce function
  • an output author

Input reader [edit]

The input reader divides the input into appropriate size 'splits' (in practice, typically, 64 MB to 128 MB) and the framework assigns one carve up to each Map function. The input reader reads data from stable storage (typically, a distributed file system) and generates fundamental/value pairs.

A mutual example will read a directory total of text files and return each line every bit a tape.

Map function [edit]

The Map function takes a series of primal/value pairs, processes each, and generates zero or more output key/value pairs. The input and output types of the map tin exist (and often are) dissimilar from each other.

If the application is doing a word count, the map function would pause the line into words and output a key/value pair for each word. Each output pair would contain the word as the key and the number of instances of that give-and-take in the line equally the value.

Sectionalization function [edit]

Each Map function output is allocated to a detail reducer by the application's partition function for sharding purposes. The partition function is given the primal and the number of reducers and returns the index of the desired reducer.

A typical default is to hash the key and use the hash value modulo the number of reducers. It is of import to selection a partition function that gives an approximately compatible distribution of data per shard for load-balancing purposes, otherwise the MapReduce performance can be held up waiting for slow reducers to finish (i.e. the reducers assigned the larger shares of the non-uniformly partitioned data).

Between the map and reduce stages, the information are shuffled (parallel-sorted / exchanged between nodes) in society to move the information from the map node that produced them to the shard in which they will be reduced. The shuffle can sometimes accept longer than the ciphering time depending on network bandwidth, CPU speeds, data produced and time taken by map and reduce computations.

Comparison part [edit]

The input for each Reduce is pulled from the machine where the Map ran and sorted using the application's comparison function.

Reduce part [edit]

The framework calls the awarding's Reduce function once for each unique key in the sorted social club. The Reduce can iterate through the values that are associated with that primal and produce zero or more outputs.

In the word count case, the Reduce function takes the input values, sums them and generates a single output of the word and the final sum.

Output writer [edit]

The Output Writer writes the output of the Reduce to the stable storage.

Operation considerations [edit]

MapReduce programs are non guaranteed to be fast. The main benefit of this programming model is to exploit the optimized shuffle performance of the platform, and only having to write the Map and Reduce parts of the program. In practice, the author of a MapReduce program however has to accept the shuffle step into consideration; in particular the partition office and the amount of data written past the Map function can have a large affect on the performance and scalability. Additional modules such every bit the Combiner role can help to reduce the corporeality of data written to deejay, and transmitted over the network. MapReduce applications can achieve sub-linear speedups nether specific circumstances.[xviii]

When designing a MapReduce algorithm, the author needs to cull a adept tradeoff[11] between the computation and the communication costs. Communication cost ofttimes dominates the computation price,[11] [18] and many MapReduce implementations are designed to write all communication to distributed storage for crash recovery.

In tuning performance of MapReduce, the complication of mapping, shuffle, sorting (grouping past the key), and reducing has to exist taken into account. The amount of data produced by the mappers is a central parameter that shifts the bulk of the computation toll between mapping and reducing. Reducing includes sorting (grouping of the keys) which has nonlinear complexity. Hence, pocket-size partition sizes reduce sorting time, but there is a trade-off because having a large number of reducers may be impractical. The influence of dissever unit size is marginal (unless chosen particularly badly, say <1MB). The gains from some mappers reading load from local disks, on average, is small.[19]

For processes that complete speedily, and where the information fits into primary retentiveness of a unmarried auto or a modest cluster, using a MapReduce framework unremarkably is non effective. Since these frameworks are designed to recover from the loss of whole nodes during the ciphering, they write interim results to distributed storage. This crash recovery is expensive, and only pays off when the ciphering involves many computers and a long runtime of the computation. A task that completes in seconds tin can just be restarted in the case of an error, and the likelihood of at least one machine failing grows quickly with the cluster size. On such bug, implementations keeping all data in memory and simply restarting a computation on node failures or —when the data is small enough— non-distributed solutions will often exist faster than a MapReduce system.

Distribution and reliability [edit]

MapReduce achieves reliability by parceling out a number of operations on the set up of data to each node in the network. Each node is expected to report back periodically with completed piece of work and status updates. If a node falls silent for longer than that interval, the chief node (similar to the main server in the Google File Organization) records the node as dead and sends out the node'due south assigned work to other nodes. Individual operations employ atomic operations for naming file outputs as a check to ensure that there are non parallel conflicting threads running. When files are renamed, it is possible to also copy them to another name in improver to the proper name of the chore (assuasive for side-effects).

The reduce operations operate much the same fashion. Because of their inferior properties with regard to parallel operations, the primary node attempts to schedule reduce operations on the same node, or in the same rack as the node property the data beingness operated on. This belongings is desirable as information technology conserves bandwidth across the courage network of the datacenter.

Implementations are not necessarily highly reliable. For example, in older versions of Hadoop the NameNode was a unmarried indicate of failure for the distributed filesystem. Later versions of Hadoop have high availability with an active/passive failover for the "NameNode."

Uses [edit]

MapReduce is useful in a broad range of applications, including distributed pattern-based searching, distributed sorting, web link-graph reversal, Singular Value Decomposition,[20] web access log stats, inverted index structure, document clustering, machine learning,[21] and statistical machine translation. Moreover, the MapReduce model has been adjusted to several computing environments like multi-core and many-core systems,[22] [23] [24] desktop grids,[25] multi-cluster,[26] volunteer calculating environments,[27] dynamic deject environments,[28] mobile environments,[29] and high-operation computing environments.[30]

At Google, MapReduce was used to completely regenerate Google'southward index of the World Wide Spider web. It replaced the one-time ad hoc programs that updated the index and ran the various analyses.[31] Development at Google has since moved on to technologies such as Percolator, FlumeJava[32] and MillWheel that offer streaming operation and updates instead of batch processing, to allow integrating "alive" search results without rebuilding the complete index.[33]

MapReduce'due south stable inputs and outputs are usually stored in a distributed file system. The transient data are normally stored on local disk and fetched remotely by the reducers.

Criticism [edit]

Lack of novelty [edit]

David DeWitt and Michael Stonebraker, figurer scientists specializing in parallel databases and shared-nothing architectures, have been critical of the breadth of problems that MapReduce tin be used for.[34] They called its interface too low-level and questioned whether it actually represents the paradigm shift its proponents accept claimed it is.[35] They challenged the MapReduce proponents' claims of novelty, citing Teradata as an instance of prior art that has existed for over two decades. They also compared MapReduce programmers to CODASYL programmers, noting both are "writing in a low-level language performing low-level tape manipulation."[35] MapReduce's use of input files and lack of schema support prevents the performance improvements enabled by common database system features such as B-trees and hash partitioning, though projects such as Pig (or PigLatin), Sawzall, Apache Hive,[36] HBase[37] and Bigtable[37] [38] are addressing some of these problems.

Greg Jorgensen wrote an article rejecting these views.[39] Jorgensen asserts that DeWitt and Stonebraker's entire assay is groundless every bit MapReduce was never designed nor intended to be used every bit a database.

DeWitt and Stonebraker have subsequently published a detailed benchmark study in 2009 comparing functioning of Hadoop's MapReduce and RDBMS approaches on several specific problems.[40] They concluded that relational databases offer real advantages for many kinds of data employ, especially on circuitous processing or where the information is used across an enterprise, but that MapReduce may be easier for users to adopt for simple or old processing tasks.

The MapReduce programming prototype was too described in Danny Hillis'south 1985 thesis [41] and was widely used at the time to plan the Connection Machine, which had special hardware back up to advance both map and reduce.

In 2010 Google was granted what is described equally a patent on MapReduce. The patent, filed in 2004, may encompass utilize of MapReduce by open source software such as Hadoop, CouchDB, and others. In Ars Technica, an editor acknowledged Google's role in popularizing the MapReduce concept, but questioned whether the patent was valid or novel.[42] [43] In 2013, every bit part of its "Open Patent Non-Exclamation (OPN) Pledge", Google pledged to only utilize the patent defensively.[44] [45] The patent is expected to expire on 23 December 2026.[46]

Restricted programming framework [edit]

MapReduce tasks must exist written as acyclic dataflow programs, i.e. a stateless mapper followed by a stateless reducer, that are executed by a batch task scheduler. This paradigm makes repeated querying of datasets difficult and imposes limitations that are felt in fields such as auto learning, where iterative algorithms that revisit a single working gear up multiple times are the norm.[47]

See also [edit]

  • Bird–Meertens formalism

Implementations of MapReduce [edit]

  • Apache CouchDB
  • Apache Hadoop
  • Infinispan
  • Riak

References [edit]

  1. ^ "MapReduce Tutorial". Apache Hadoop . Retrieved 3 July 2019.
  2. ^ "Google spotlights data center inner workings". cnet.com. thirty May 2008.
  3. ^ "MapReduce: Simplified Information Processing on Big Clusters" (PDF). googleusercontent.com.
  4. ^ Wickham, Hadley (2011). "The split-apply-combine strategy for data analysis". Journal of Statistical Software. 40: one–29. doi:10.18637/jss.v040.i01.
  5. ^ "Our abstraction is inspired by the map and reduce primitives present in Lisp and many other functional languages." -"MapReduce: Simplified Data Processing on Large Clusters", by Jeffrey Dean and Sanjay Ghemawat; from Google Research
  6. ^ Lämmel, R. (2008). "Google'south Map Reduce programming model — Revisited". Science of Figurer Programming. 70: 1–xxx. doi:10.1016/j.scico.2007.07.001.
  7. ^ http://world wide web.mcs.anl.gov/inquiry/projects/mpi/mpi-standard/mpi-report-2.0/mpi2-report.htm MPI ii standard
  8. ^ "MPI Reduce and Allreduce · MPI Tutorial". mpitutorial.com.
  9. ^ "Performing Parallel Rank with MPI · MPI Tutorial". mpitutorial.com.
  10. ^ "MongoDB: Terrible MapReduce Functioning". Stack Overflow. October 16, 2010. The MapReduce implementation in MongoDB has little to do with map reduce patently. Because for all I read, it is single-threaded, while map-reduce is meant to be used highly parallel on a cluster. ... MongoDB MapReduce is single threaded on a unmarried server...
  11. ^ a b c Ullman, J. D. (2012). "Designing good MapReduce algorithms". XRDS: Crossroads, the ACM Magazine for Students. 19: thirty–34. doi:x.1145/2331042.2331053. S2CID 26498063.
  12. ^ Sverdlik, Yevgeniy (2014-06-25). "Google Dumps MapReduce in Favor of New Hyper-Scale Analytics System". Data Center Knowledge . Retrieved 2015-10-25 . "We don't actually use MapReduce anymore" [Urs Hölzle, senior vice president of technical infrastructure at Google]
  13. ^ Harris, Derrick (2014-03-27). "Apache Mahout, Hadoop's original machine learning project, is moving on from MapReduce". Gigaom . Retrieved 2015-09-24 . Apache Mahout [...] is joining the exodus away from MapReduce.
  14. ^ Czajkowski, Grzegorz; Marián Dvorský; Jerry Zhao; Michael Conley. "Sorting Petabytes with MapReduce – The Adjacent Episode". Retrieved 7 April 2014.
  15. ^ "MapReduce Tutorial".
  16. ^ "Apache/Hadoop-mapreduce". GitHub. 31 Baronial 2021.
  17. ^ "Instance: Count give-and-take occurrences". Google Research. Retrieved September 18, 2013.
  18. ^ a b Senger, Hermes; Gil-Costa, Veronica; Arantes, Luciana; Marcondes, Cesar A. C.; Marín, Mauricio; Sato, Liria Yard.; da Silva, Fabrício A.B. (2015-01-01). "BSP cost and scalability analysis for MapReduce operations". Concurrency and Ciphering: Practise and Experience. 28 (8): 2503–2527. doi:10.1002/cpe.3628. hdl:10533/147670. ISSN 1532-0634. S2CID 33645927.
  19. ^ Berlińska, Joanna; Drozdowski, Maciej (2010-12-01). "Scheduling divisible MapReduce computations". Journal of Parallel and Distributed Computing. 71 (three): 450–459. doi:10.1016/j.jpdc.2010.12.004.
  20. ^ Bosagh Zadeh, Reza; Carlsson, Gunnar (2013). "Dimension Independent Matrix Square Using MapReduce" (PDF). arXiv:1304.1467. Bibcode:2013arXiv1304.1467B. Retrieved 12 July 2014.
  21. ^ Ng, Andrew Y.; Bradski, Gary; Chu, Cheng-Tao; Olukotun, Kunle; Kim, Sang Kyun; Lin, Yi-An; Yu, YuanYuan (2006). "Map-Reduce for Car Learning on Multicore". NIPS 2006.
  22. ^ Ranger, C.; Raghuraman, R.; Penmetsa, A.; Bradski, G.; Kozyrakis, C. (2007). "Evaluating MapReduce for Multi-core and Multiprocessor Systems". 2007 IEEE 13th International Symposium on Loftier Performance Estimator Architecture. p. 13. CiteSeerX10.1.1.220.8210. doi:x.1109/HPCA.2007.346181. ISBN978-one-4244-0804-7. S2CID 12563671.
  23. ^ He, B.; Fang, Due west.; Luo, Q.; Govindaraju, North. K.; Wang, T. (2008). "Mars: a MapReduce framework on graphics processors" (PDF). Proceedings of the 17th international conference on Parallel architectures and compilation techniques – PACT '08. p. 260. doi:x.1145/1454115.1454152. ISBN9781605582825. S2CID 207169888.
  24. ^ Chen, R.; Chen, H.; Zang, B. (2010). "Tiled-MapReduce: optimizing resource usages of data-parallel applications on multicore with tiling". Proceedings of the 19th international conference on Parallel architectures and compilation techniques – PACT '10. p. 523. doi:x.1145/1854273.1854337. ISBN9781450301787. S2CID 2082196.
  25. ^ Tang, B.; Moca, K.; Chevalier, S.; He, H.; Fedak, M. (2010). "Towards MapReduce for Desktop Filigree Calculating" (PDF). 2010 International Conference on P2P, Parallel, Grid, Cloud and Cyberspace Computing. p. 193. CiteSeerXx.one.1.671.2763. doi:10.1109/3PGCIC.2010.33. ISBN978-one-4244-8538-three. S2CID 15044391.
  26. ^ Luo, Y.; Guo, Z.; Sun, Y.; Plale, B.; Qiu, J.; Li, W. (2011). "A Hierarchical Framework for Cross-Domain MapReduce Execution" (PDF). Proceedings of the second international workshop on Emerging computational methods for the life sciences (ECMLS '11). CiteSeerX10.one.1.364.9898. doi:10.1145/1996023.1996026. ISBN978-ane-4503-0702-4. S2CID 15179363.
  27. ^ Lin, H.; Ma, X.; Archuleta, J.; Feng, West. C.; Gardner, M.; Zhang, Z. (2010). "MOON: MapReduce On Opportunistic eNvironments" (PDF). Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing – HPDC '10. p. 95. doi:10.1145/1851476.1851489. ISBN9781605589428. S2CID 2351790.
  28. ^ Marozzo, F.; Talia, D.; Trunfio, P. (2012). "P2P-MapReduce: Parallel data processing in dynamic Cloud environments". Periodical of Computer and Organisation Sciences. 78 (5): 1382–1402. doi:10.1016/j.jcss.2011.12.021.
  29. ^ Dou, A.; Kalogeraki, Five.; Gunopulos, D.; Mielikainen, T.; Tuulos, V. H. (2010). "Misco: a MapReduce framework for mobile systems". Proceedings of the 3rd International Conference on PErvasive Technologies Related to Assistive Environments – PETRA 'x. p. 1. doi:10.1145/1839294.1839332. ISBN9781450300711. S2CID 14517696.
  30. ^ Wang, Yandong; Goldstone, Robin; Yu, Weikuan; Wang, Teng (May 2014). "Characterization and Optimization of Memory-Resident MapReduce on HPC Systems". 2014 IEEE 28th International Parallel and Distributed Processing Symposium. IEEE. pp. 799–808. doi:10.1109/IPDPS.2014.87. ISBN978-i-4799-3800-1. S2CID 11157612.
  31. ^ "How Google Works". baselinemag.com. Equally of October, Google was running virtually 3,000 computing jobs per day through MapReduce, representing thousands of machine-days, co-ordinate to a presentation past Dean. Among other things, these batch routines analyze the latest Web pages and update Google'southward indexes.
  32. ^ Chambers, Craig; Raniwala, Ashish; Perry, Frances; Adams, Stephen; Henry, Robert R.; Bradshaw, Robert; Weizenbaum, Nathan (ane January 2010). FlumeJava: Easy, Efficient Data-parallel Pipelines (PDF). Proceedings of the 31st ACM SIGPLAN Conference on Programming Language Blueprint and Implementation. pp. 363–375. doi:10.1145/1806596.1806638. ISBN9781450300193. S2CID 14888571. Archived from the original (PDF) on 23 September 2016. Retrieved 4 August 2016.
  33. ^ Peng, D., & Dabek, F. (2010, October). Large-scale Incremental Processing Using Distributed Transactions and Notifications. In OSDI (Vol. x, pp. 1-15).
  34. ^ "Database Experts Jump the MapReduce Shark".
  35. ^ a b David DeWitt; Michael Stonebraker. "MapReduce: A major step backwards". craig-henderson.blogspot.com. Retrieved 2008-08-27 .
  36. ^ "Apache Hive – Alphabetize of – Apache Software Foundation".
  37. ^ a b "HBase – HBase Home – Apache Software Foundation".
  38. ^ "Bigtable: A Distributed Storage System for Structured Information" (PDF).
  39. ^ Greg Jorgensen. "Relational Database Experts Jump The MapReduce Shark". typicalprogrammer.com. Retrieved 2009-11-11 .
  40. ^ Pavlo, Andrew; Paulson, Erik; Rasin, Alexander; Abadi, Daniel J.; DeWitt, Deavid J.; Madden, Samuel; Stonebraker, Michael. "A Comparison of Approaches to Large-Scale Information Analysis". Brown Academy. Retrieved 2010-01-11 .
  41. ^ Hillis, W. Danny (1986). The Connection Machine . MIT Press. ISBN0262081571.
  42. ^ Paul, Ryan (20 January 2010). "Google's MapReduce patent: what does information technology mean for Hadoop?". Ars Technica . Retrieved 21 March 2021.
  43. ^ "The states Patent: 7650331 - Organization and method for efficient large-calibration data processing". uspto.gov.
  44. ^ Nazer, Daniel (28 March 2013). "Google Makes Open Patent Non-exclamation Pledge and Proposes New Licensing Models". Electronic Frontier Foundation . Retrieved 21 March 2021.
  45. ^ King, Rachel (2013). "Google expands open up patent pledge to 79 more almost information center direction". ZDNet . Retrieved 21 March 2021.
  46. ^ "Organisation and method for efficient large-scale data processing". Google Patents Search. eighteen June 2004. Retrieved 21 March 2021.
  47. ^ Zaharia, Matei; Chowdhury, Mosharaf; Franklin, Michael; Shenker, Scott; Stoica, Ion (June 2010). Spark: Cluster Calculating with Working Sets. HotCloud 2010.

DOWNLOAD HERE

Posted by: falknerinece1940.blogspot.com

Post a Comment

Post a Comment

Previous Post Next Post

Iklan Banner setelah judul