cat.lump.ie.textprocessing.stopwords

Class Stopwords

java.lang.Object

cat.lump.ie.textprocessing.stopwords.Stopwords

public class Stopwords
extends java.lang.Object

Abstract class that gives the methods for stopwords acquisition and modification in different languages. The comparison strategies are case sensitive.

Author:

albarron \\TODO Ull! now nbsp is removed even if it is not in STOP_LIST

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.List<java.lang.String>	STOP_LIST

Constructor Summary

Constructors

Constructor and Description
Stopwords(java.util.Locale language)

Method Summary

Methods

Modifier and Type	Method and Description
java.util.List<java.lang.String>	getStopWords()
boolean	isStopword(java.lang.String word)
void	removeStopwords(java.util.List<java.lang.String> words)
java.lang.String	removeStopwords(java.lang.String text) Checks for the vocabulary in the text and returns a copy after discarding stopwords.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

STOP_LIST

protected java.util.List<java.lang.String> STOP_LIST

Constructor Detail

Stopwords

public Stopwords(java.util.Locale language)

Method Detail

getStopWords

public java.util.List<java.lang.String> getStopWords()

Returns:

list of stopwords

isStopword

public boolean isStopword(java.lang.String word)

Parameters:

word -

Returns:

true if the word is a stopword

removeStopwords

public java.lang.String removeStopwords(java.lang.String text)

Checks for the vocabulary in the text and returns a copy after discarding stopwords. The text is expected to be previously tokenized.

Parameters:

text -

Returns:

text with stopwords discarded

removeStopwords

public void removeStopwords(java.util.List<java.lang.String> words)