Criado por Nilesh Patel
mais de 6 anos atrás
|
||
Word Count: Example 1 (RDD) val txtfile=sc.textFile("C:/Users/Redirection/pateln7/Downloads/SPARK/wordcount.txt") val txtflatmap= txtfile.flatMap(x=>x.split(" ")) txtflatmap.first val txtmap=txtflatmap.map(x=>(x,1)) txtmap.first val output=txtmap.reduceByKey(_+_) output.foreach(println) Single line : val output = txtfile.flatMap(x=>x.split(" ")).map(x=>(x,1)).reduceByKey(_+_) output.foreach(println) Example 2 (RDD) val txtfile=sc.textFile("C:/Users/Redirection/pateln7/Downloads/SPARK/wordcount.txt") val txtflatmap= txtfile.flatMap(x=>x.split(" ")) txtflatmap.first val txtmap=txtflatmap.map(x=>(x,1)) txtmap.first val output = txtmap.reduceByKey((x,txtmap)=>(x+txtmap)).foreach(println) Example 3 (for Array[(string ,Int) ] type data val str = "Hell saying hello is good, keep say hello" val x =str.split(" ").map(x=> (x, 1)) val q = sc.parallelize(x) q.reduceByKey(_+_).foreach(println)
Quer criar suas próprias Notas gratuitas com a GoConqr? Saiba mais.