想做个层次聚类,转置后设置行名时提示Error in `rownames<-`(`*tmp*`, value = c(271L, 247L, 342L, 1158L, 1338L, :
length of 'dimnames' [1] not equal to array extent
plot的时候又提示
Error in graphics:::plotHclust(n1, merge, height, order(x$order), hang, :
谱系图输入不对
以下是代码和数据,望各位大神帮忙看看,在下新手,多谢多谢!
> csv <- read.table("body1.txt",header=T)
> mystopwords<- unlist (read.table("StopWords.txt",stringsAsFactors=F))
> library(tm)
> #移除数字
> removeNumbers = function(x) { ret = gsub("[0-90123456789]","",x) }
> #中文分词,也可以考虑使用rmmseg4j、rsmartcn
> wordsegment<- function(x) {
+ library(Rwordseg)
+ segmentCN(x)
+ }
> #去除停止词,效果比较差,可以进一步完善
> removeStopWords = function(x,words) {
+ ret = character(0)
+ index <- 1
+ it_max <- length(x)
+ while (index <= it_max) {
+ if (length(words[words==x[index]]) <1) ret <- c(ret,x[index])
+ index <- index +1
+ }
+ ret
+ }
> Sys.setenv(JAVA_HOME='C:/Program Files/Java/jdk1.6.0_43/jre')
> sample.words <- lapply(csv, removeNumbers)
> sample.words <- lapply(sample.words, wordsegment)
> #先处理中文分词,再处理stopwords,防止全局替换丢失信息
> sample.words <- lapply(sample.words, removeStopWords, mystopwords)
> #构建语料库
> corpus = Corpus(VectorSource(sample.words))
>
> sample.dtm <- DocumentTermMatrix(corpus, control = list(wordLengths = c(2, Inf)))
> sample_matrix =as.matrix(sample.dtm)
> rownames(sample_matrix)<- csv$newsid
Error in `rownames<-`(`*tmp*`, value = c(271L, 247L, 342L, 1158L, 1338L, :
length of 'dimnames' [1] not equal to array extent
>
> dtm2 = removeSparseTerms(sample.dtm, sparse=0.99)
> d <- dist(dtm2, method = "euclidean")
> fit <- hclust(d, method="ward.D")
> plot(fit)
Error in graphics:::plotHclust(n1, merge, height, order(x$order), hang, :
谱系图输入不对
还想问问大家,由于我的数据量比较大,做层次聚类的谱系图会太密集,看不清,有什么更好的文本聚类方法吗?我后续还想进行如用cutree提取出每条新闻属于哪一类的操作,不知道有没有别的方法或函数能做到的,多谢大家了!