我用谷歌搜索无济于事。这里有人可以告诉我如何在 Prolog 中编码词频吗？越“逻辑”越好。给定一个文本文件，忽略非字母数字字符，检测单词，删除停用词（可能在外部文件中给出），计算文件中每个单词的出现次数并输出类似“word : freq”的内容，按频率递减排序。
谢谢！
（这不是作业，顺便说一句，这是我正在做的一个项目）
问问题
982 次
3 回答
2
我在此答案中使用 SWI-Prolog 广泛的库支持，因此在您的情况下可能不合适。
当然，您解决任何编程任务的方式很大程度上受您选择的语言中可用的内容以及您使用它的能力的影响。
这里我使用library(assoc)，如果您的系统错过了它，那么您可以使用列表进行模拟，或者使用 assert/retract。
这个片段只是展示了一种（惯用的）计算单词的方法，计算频率可以通过库（聚合）或一些算术轻松完成，也许你想尝试自己写下来练习语言......
/* File: frequency_of_words.pl
Author: Carlo,,,
Created: May 23 2012
Purpose: http://stackoverflow.com/questions/10711483/calculating-term-frequency-in-prolog
*/
:- module(frequency_of_words, [count_words/2, count_words/1]).
:- [library(assoc)].
count_words(File, Assoc) :-
empty_assoc(Empty),
open(File, read, Stream),
frequency_of_words(Stream, Empty, Assoc, ""),
close(Stream).
count_words(File) :-
count_words(File, Assoc),
assoc_to_list(Assoc, List),
maplist(writeln, List).
frequency_of_words(Stream, SoFar, Words, CurrWord) :-
get_code(Stream, Code),
( Code == -1
-> update_dictionary(SoFar, Words, CurrWord)
; use_character(Code, SoFar, Updated, CurrWord, NextWord),
frequency_of_words(Stream, Updated, Words, NextWord)
).
update_dictionary(SoFar, SoFar, Word) :-
skip_word(Word).
update_dictionary(SoFar, Updated, Codes) :-
atom_codes(Word, Codes),
( get_assoc(Word, SoFar, CountSoFar) ; CountSoFar = 0 ),
WordCount is CountSoFar + 1,
put_assoc(Word, SoFar, WordCount, Updated).
use_character(Code, SoFar, Updated, CurrWord, NextWord) :-
( word_character(Code)
-> Updated = SoFar,
NextWord = [Code|CurrWord]
; reverse(CurrWord, Forward),
update_dictionary(SoFar, Updated, Forward),
NextWord = ""
).
word_character(Code) :-
[Code] @>= "A", [Code] @=< "Z" ;
[Code] @>= "a", [Code] @=< "z" ;
[Code] @>= "0", [Code] @=< "9" ;
[Code] == "_".
skip_word(""). % a trick on EOF or consecutive blanks: not really a skipword
skip_word("is").
测试：
?- count_words('frequency_of_words.pl').
0-2
1-3
2012-1
23-1
9-1
A-1
Author-1
Carlo-1
Code-14
Codes-2
CountSoFar-3
Created-1
CurrWord-6
EOF-1
Empty-2
File-3
Forward-2
Full-2
List-2
May-1
NextWord-5
Purpose-1
SoFar-11
StackOverflow-1
Stream-6
Updated-7
Word-5
WordCount-2
Words-3
Z-1
_-1
a-3
answer-1
assoc-1
assoc_to_list-1
atom_codes-1
blanks-1
close-1
consecutive-1
count_words-2
empty_assoc-1
frequency_of_words-5
get_assoc-1
get_code-1
library-1
maplist-1
module-1
not-1
on-1
open-1
or-1
pl-1
put_assoc-1
read-1
really-1
reverse-1
skip_word-3
skipword-1
trick-1
update_dictionary-4
use_character-2
word_character-2
writeln-1
z-1
于 2012-05-23T06:23:34.450 回答
2
这里有一些东西可以帮助你开始：
它假设每个单词都在输入文件的单独行上
不处理“停用词”
不实现排序
; Print the term frequency for words in file Path
print_freq_from_file(Path) :-
open( Path, read, Stream ),
read_file( Stream, Words ),
calc_freq( Words, Freq ),
sort_freq( Freq, SortedFreq ),
print_freq( SortedFreq ).
; calc_freq(Words,Freq) Freq is the term frequency of the words in list Words
calc_freq( [], [] ).
calc_freq( [Word|Words], Freq ) :-
calc_freq( Words, FreqRest ),
add_word( Word, FreqRest, Freq ).
; add_word( Word, Freq1, Freq2 ) Freq2 is Freq1 with Word added
add_word( Word, [], [(Word,1)] ).
add_word( Word, [(Word,N)|Rest], [(Word,N1)|Rest] ) :-
N1 is N+1.
add_word( Word, [Term|Rest], [Term|NewRest] ) :-
add_word( Word, Rest, NewRest ).
; Print the given term frequency
print_freq( [] ).
print_freq( [(W,N)|Rest] ) :-
write( W ), write( ' : ' ), write( N ), nl,
print_freq( Rest ).
于 2012-05-23T00:02:35.757 回答
1
对于过滤停用词，一个好方法是通过assert：
read_sw(Path) :-
open( Path, read, Stream ),
read_words_file( Stream, Words ),
add_sw(Words).
add_sw([]).
add_sw([Word|Rest]) :-
assertz(stop_word(Word)),
add_sw(Rest).
完成此操作后，您可以使用stop_word/1检查单词是否为停用词并忽略它。例如，如果您正在实施 Scott Hunter 的答案，则可以添加一个附加子句calc_freq/2来过滤掉这些词：
calc_freq( [], [] ).
calc_freq( [Word|Words], Freq ) :-
stop_word(Word), !,
calc_freq( Words, Freq ).
calc_freq( [Word|Words], Freq ) :-
calc_freq( Words, FreqRest ),
add_word( Word, FreqRest, Freq ).
于 2012-05-23T05:35:26.310 回答