① ikanalyzer 可以做詞性標注嗎
可以的
這兩天正好在玩lucene,沒用庖丁分詞,主要是嫌它要配置環境,麻煩
下面是demo,記得要加lucene-core-2.3.2.jar和lucene-Analyzer.jar以及IKAnalyzer.jar這幾個包,有問題call我
import java.io.Reader;
import java.io.StringReader;
import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.analysis.StopFilter;
import org.apache.lucene.analysis.Token;
import org.apache.lucene.analysis.TokenFilter;
import org.apache.lucene.analysis.TokenStream;
import org.apache.lucene.analysis.cjk.CJKAnalyzer;
import org.apache.lucene.analysis.cn.ChineseAnalyzer;
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.mira.lucene.analysis.IK_CAnalyzer;
public class TestJeAnalyzer {
private static String testString1 = "冗長的代碼常常是復雜性的標志，會導致代碼難以測試和維護.";
public static void testStandard(String testString) throws Exception{
Analyzer analyzer = new StandardAnalyzer();
Reader r = new StringReader(testString);
StopFilter sf = (StopFilter) analyzer.tokenStream("", r);
System.err.println("=====standard analyzer====");
System.err.println("分析方法：默認沒有詞只有字");
Token t;
while ((t = sf.next()) != null) {
System.out.println(t.termText());
}
}
public static void testCJK(String testString) throws Exception{
Analyzer analyzer = new CJKAnalyzer();
Reader r = new StringReader(testString);
StopFilter sf = (StopFilter) analyzer.tokenStream("", r);
System.err.println("=====cjk analyzer====");
System.err.println("分析方法:交叉雙字分割");
Token t;
while ((t = sf.next()) != null) {
System.out.println(t.termText());
}
}
public static void testChiniese(String testString) throws Exception{
Analyzer analyzer = new ChineseAnalyzer();
Reader r = new StringReader(testString);
TokenFilter tf = (TokenFilter) analyzer.tokenStream("", r);
System.err.println("=====chinese analyzer====");
System.err.println("分析方法:基本等同StandardAnalyzer");
Token t;
while ((t = tf.next()) != null) {
System.out.println(t.termText());
}
}
public static void testJe(String testString) throws Exception{
// Analyzer analyzer = new MIK_CAnalyzer();
Analyzer analyzer = new IK_CAnalyzer();
Reader r = new StringReader(testString);
TokenStream ts = (TokenStream)analyzer.tokenStream("", r);
System.err.println("=====je analyzer====");
System.err.println("分析方法:字典分詞,正反雙向搜索，具體不明");
Token t;
while ((t = ts.next()) != null) {
System.out.println(t.termText());
}
}
public static void main(String[] args) throws Exception{
// String testString = testString1;
String testString = testString1;
System.out.println(testString);
testStandard(testString);
testCJK(testString);
// testPaoding(testString);
testChiniese(testString);
testJe(testString);
}
}
③ 誰來推薦一個JAVA的分詞工具
java讀取中文分詞工具：linger
Java開源中文分詞器
1、word分詞器
2、Ansj分詞器
3、Stanford分詞器
4、FudanNLP分詞器
5、Jieba分詞器
6、Jcseg分詞器
7、MMSeg4j分詞器
8、IKAnalyzer分詞器
9、Paoding分詞器
10、smartcn分詞器