Word Count:

Example 1 (RDD)

val txtfile=sc.textFile("C:/Users/Redirection/pateln7/Downloads/SPARK/wordcount.txt")

val txtflatmap= txtfile.flatMap(x=>x.split(" "))

txtflatmap.first

val txtmap=txtflatmap.map(x=>(x,1))

txtmap.first

val output=txtmap.reduceByKey(_+_)

output.foreach(println)

Single line :

val output = txtfile.flatMap(x=>x.split(" ")).map(x=>(x,1)).reduceByKey(_+_)

output.foreach(println)

Example 2 (RDD)

val txtfile=sc.textFile("C:/Users/Redirection/pateln7/Downloads/SPARK/wordcount.txt")

val txtflatmap= txtfile.flatMap(x=>x.split(" "))

txtflatmap.first

val txtmap=txtflatmap.map(x=>(x,1))

txtmap.first

val output = txtmap.reduceByKey((x,txtmap)=>(x+txtmap)).foreach(println)

Example 3 (for Array[(string ,Int) ] type data

val str = "Hell saying hello is good, keep say hello"

val x =str.split(" ").map(x=> (x, 1))

val q = sc.parallelize(x)

q.reduceByKey(_+_).foreach(println)

	Erstellt von Nilesh Patel vor mehr als 6 Jahre

WordCount