这就是我为您的数据建模的方式(请注意,“the”被引用了两次)您还可以为单个单词添加权重。
DROP SCHEMA ngram CASCADE;
CREATE SCHEMA ngram;
SET search_path='ngram';
CREATE table word
( word_id INTEGER PRIMARY KEY
, the_word varchar
, constraint word_the_word UNIQUE (the_word)
);
CREATE table ngram
( ngram_id INTEGER PRIMARY KEY
, n INTEGER NOT NULL -- arity
, weight REAL -- payload
);
CREATE TABLE ngram_word
( ngram_id INTEGER NOT NULL REFERENCES ngram(ngram_id)
, seq INTEGER NOT NULL
, word_id INTEGER NOT NULL REFERENCES word(word_id)
, PRIMARY KEY (ngram_id,seq)
);
INSERT INTO word(word_id,the_word) VALUES
(1, 'the') ,(2, 'man') ,(3, 'who') ,(4, 'sold') ,(5, 'world' );
INSERT INTO ngram(ngram_id, n, weight) VALUES
(101, 6, 1.0);
INSERT INTO ngram_word(ngram_id,seq,word_id) VALUES
( 101, 1, 1)
, ( 101, 2, 2)
, ( 101, 3, 3)
, ( 101, 4, 4)
, ( 101, 5, 1)
, ( 101, 6, 5)
;
SELECT w.*
FROM ngram_word nw
JOIN word w ON w.word_id = nw.word_id
WHERE ngram_id = 101
ORDER BY seq;
结果:
word_id | the_word
---------+----------
1 | the
2 | man
3 | who
4 | sold
1 | the
5 | world
(6 rows)
现在,假设您想在现有(6 克)数据中添加一个 4 克:
INSERT INTO word(word_id,the_word) VALUES
(6, 'is') ,(7, 'lost') ;
INSERT INTO ngram(ngram_id, n, weight) VALUES
(102, 4, 0.1);
INSERT INTO ngram_word(ngram_id,seq,word_id) VALUES
( 102, 1, 1)
, ( 102, 2, 2)
, ( 102, 3, 6)
, ( 102, 4, 7)
;
SELECT w.*
FROM ngram_word nw
JOIN word w ON w.word_id = nw.word_id
WHERE ngram_id = 102
ORDER BY seq;
附加结果:
INSERT 0 2
INSERT 0 1
INSERT 0 4
word_id | the_word
---------+----------
1 | the
2 | man
6 | is
7 | lost
(4 rows)
顺便说一句:向这个模型添加一个文档类型对象将向这个模型添加两个额外的表:一个用于文档,一个用于 document*ngram。(或以另一种方法:对于文档*单词)递归模型也是一种可能性。
更新:上述模型将需要一个额外的约束,这将需要实现触发器(或规则+一个额外的表)。伪代码:
ngram_word.seq >0 AND ngram_word.seq <= (select ngram.n FROM ngram ng WHERE ng.ngram_id = ngram_word.ngram_id)