2

我的目标是从网页中提取和解析一系列参考书目,以便稍后输入数据库。参考文献均采用 MLA 格式。对于 MLA 格式书目的所有实例,这应该是一个通用解决方案,并且应该不仅仅适用于下面所示的网页。

这是我的尝试代码,它不起作用:

(use '[net.cgrand.enlive-html])

(def ^:dynamic *base-url* "https://www.impacttest.com/research/?Clinical-Research-Database-4")
(def ^:dynamic *ref-selector*     [:div#content_1 :ul :li])


(defn fetch-url [url]
  (html-resource (java.net.URL. url)))

(defn references []
  (select (fetch-url *base-url*) *ref-selector*))

(def ^:dynamic *ref-regex*    #"\s([A-Z]{1}[\w|\s]+)[,|\.]")
(def ^:dynamic *ref-modifier* `(remove :content))

(defmacro extract-re [node re modifier]
  `(doseq [seqs (map :content (node))]
    (re-find re (apply str (modifier seqs)))))

(extract-re references *ref-regex* *ref-modifier*)

(macroexpand-1 '(extract-re references *ref-regex* *ref-modifier*))

我希望宏extract-re创建一个在所有活动节点上doseq运行正则表达式匹配器 ( ) 的宏。re-find有两个变量需要更改:一个是正则表达式本身,另一个是修饰符,它在处理激活节点之前对其进行修改。如果没有修饰符,正则表达式将匹配作者某些标题。我尝试编写一个函数,但不能让它在一般情况下工作,所以我认为宏是要走的路。

在 MLA 引用中,我认为在激活节点上使用修饰符比使用正则表达式进行所有提取更容易,尽管我可能错了。我想不出如何做一个只匹配标题或只匹配作者的正则表达式。

那么,如何将修饰符传递给宏并使其正确执行?我不完全理解宏的引用细节,所以我可能对我是如何编写宏开始的,或者即使宏是必要的。

4

2 回答 2

3

这段代码有很多问题。

'(use [net.cgrand.enlive-html])

这不会引入库,它会创建一个文字列表,并且什么都不做:

user> (class '(use [net.cgrand.enlive-html]))
clojure.lang.PersistentList

它实际上是一个空操作。

(def ^:dynamic *ref-modifier* `(remove :content))

这会创建一个包含两个元素的列表,而不是任何类型的“修饰符”。

(defmacro extract-re [node re modifier]
  `(doseq [seqs (map :content (node))]
    (re-find re (apply str (modifier seqs)))))

在这里你使用语法引用,但你从不取消引用其中的任何内容。宏不以任何方式使用它的任何参数。

您似乎想像modifier一个函数一样应用它(这甚至没有开始发生,请参阅上面的引用问题),但正如我们在实际调用中看到的那样,modifier它是一个两元素列表,如果调用会导致错误.

最后,doseq仅适用于副作用,并且始终返回 nil。doseq 块不使用 生成的值re-find,所以 doseq 体实际上是一个空操作。

此外,我发现在将动态 var 声明用于将作为显式函数参数提供的 var 中的实用程序令人怀疑。

解决所有这些问题后,我认为我们更接近可行的方法:

(use 'net.cgrand.enlive-html)

(def ^:dynamic *base-url*
  "https://www.impacttest.com/research/?Clinical-Research-Database-4")

(def ^:dynamic *ref-selector* [:div#content_1 :ul :li])


(defn fetch-url [url]
  (html-resource (java.net.URL. url)))

(defn references []
  (select (fetch-url *base-url*) *ref-selector*))

(def ^:dynamic *ref-regex* #"\s([A-Z]{1}[\w|\s]+)[,|\.]")

(def ^:dynamic *ref-modifier* (partial remove :content))

(defn extract-re [node re modifier]
  (doall
    (for [sq (map :content (node))]
      (re-find re (apply str (modifier sq))))))

并在行动中:

user> (extract-re references *ref-regex* *ref-modifier*)

([" Dambinova SA," "Dambinova SA"] [" Zuckerman SL," "Zuckerman SL"] [" Conklin HM," "Conklin HM"] [" Covassin T," "Covassin T"] [" Maerlender A," "Maerlender A"] [" Fedor A," "Fedor A"] [" Resch J," "Resch J"] [" Elbin RJ," "Elbin RJ"] [" Rabinowitz AR," "Rabinowitz AR"] [" Kinnaman KA," "Kinnaman KA"] [" Tsushima WT," "Tsushima WT"] [" Amonette WE," "Amonette WE"] [" Lovell MR," "Lovell MR"] [" Schatz P," "Schatz P"] [" McGrath N," "McGrath N"] [" Kontos AP," "Kontos AP"] [" AB," "AB"] [" Meehan WP," "Meehan WP"] [" Rieger BP," "Rieger BP"] [" Solomon GS," "Solomon GS"] [" Sandel NK," "Sandel NK"] [" Schatz P," "Schatz P"] [" Schatz P," "Schatz P"] [" Lebrun CM," "Lebrun CM"] [" Brooks B," "Brooks B"] [" Meehan WP," "Meehan WP"] [" Fakhran S," "Fakhran S"] [" Cole WR," "Cole WR"] [" Tsushima M," "Tsushima M"] [" Zuckerman SL," "Zuckerman SL"] [" JK," "JK"] [" Covassin T," "Covassin T"] [" Moser RS," "Moser RS"] [" Mayers LB," "Mayers LB"] [" McAllister TW," "McAllister TW"] [" Meehan WP 3rd," "Meehan WP 3rd"] [" Neal MT," "Neal MT"] [" Lau BC," "Lau BC"] [" Kontos AP," "Kontos AP"] [" Gardner A," "Gardner A"] [" Elbin RJ," "Elbin RJ"] [" Wolf EG," "Wolf EG"] [" Reddy CC," "Reddy CC"] [" Moser RS," "Moser RS"] [" Guerriero RM," "Guerriero RM"] [" Deibert E," "Deibert E"] [" Wiebe DJ," "Wiebe DJ"] [" Baillargeon A," "Baillargeon A"] [" Erdal K." "Erdal K"] [" Maugans TA," "Maugans TA"] [" Iverson GL," "Iverson GL"] [" Ponsford J," "Ponsford J"] [" Schatz P," "Schatz P"] [" Mulligan I," "Mulligan I"] [" Echlin PS," "Echlin PS"] [" McLeod TC," "McLeod TC"] [" Zuckerman SL," "Zuckerman SL"] [" Kontos AP," "Kontos AP"] [" Zuckerman SL," "Zuckerman SL"] [" Schatz P," "Schatz P"] [" Kontos AP," "Kontos AP"] [" Covassin T," "Covassin T"] [" Covassin T," "Covassin T"] [" Duhaime AC," "Duhaime AC"] [" Echemendia RJ," "Echemendia RJ"] [" Ramanathan DM," "Ramanathan DM"] [" Meehan WP 3rd," "Meehan WP 3rd"] [" Krol AL," "Krol AL"] [" Turgeon C," "Turgeon C"] [" Randolph C." "Randolph C"] [" Barlow M," "Barlow M"] [" Schatz P," "Schatz P"] [" Moser RS," "Moser RS"] [" Broglio SP," "Broglio SP"] [" Thomas DG," "Thomas DG"] [" Allen BJ," "Allen BJ"] [" Solomon GS," "Solomon GS"] [" Ponsford J," "Ponsford J"] [" Johnson EW," "Johnson EW"] [" Randolph C," "Randolph C"] [" Elbin RJ," "Elbin RJ"] [" Broglio SP," "Broglio SP"] [" Kontos AP," "Kontos AP"] [" Lau BC," "Lau BC"] [" Lau BC," "Lau BC"] [" Hettich T," "Hettich T"] [" Elbin T," "Elbin T"] [" Maerlender A," "Maerlender A"] [" Kontos AP," "Kontos AP"] [" Talavage TM," "Talavage TM"] [" Meehan WP 3rd," "Meehan WP 3rd"] [" Lange RT," "Lange RT"] [" Covassin T," "Covassin T"] [" Schatz P." "Schatz P"] [" Lange RT," "Lange RT"] [" Pardini JE," "Pardini JE"] [" Echlin PS," "Echlin PS"] [" Schatz P," "Schatz P"] [" Echlin PS," "Echlin PS"] [" Keightley ML," "Keightley ML"] [" McGrath N." "McGrath N"] [" Covassin T," "Covassin T"] [" Pontifex MB," "Pontifex MB"] [" AB," "AB"] [" Casson IR," "Casson IR"] [" McCrory P," "McCrory P"] [" Covassin T," "Covassin T"] [" Bruce JM," "Bruce JM"] [" Covassin T," "Covassin T"] [" Lovell M." "Lovell M"] [" Lau B," "Lau B"] [" Nance ML," "Nance ML"] [" Peterson SE," "Peterson SE"] [" Lovell M." "Lovell M"] [" Broglio SP," "Broglio SP"] [" Broglio SP," "Broglio SP"] [" Colvin AC," "Colvin AC"] [" Reddy CC," "Reddy CC"] [" Solomon GS," "Solomon GS"] [" Covassin T," "Covassin T"] [" Majerske CW," "Majerske CW"] [" Lovell MR," "Lovell MR"] [" AB," "AB"] [" Tsushima WT," "Tsushima WT"] [" Miller JR," "Miller JR"] [" Slobounov S," "Slobounov S"] [" Mihalik JP," "Mihalik JP"] [" Covassin T," "Covassin T"] [" Lovell MR," "Lovell MR"] [" Stoller KP." "Stoller KP"] [" Broglio SP," "Broglio SP"] [" Moser RS," "Moser RS"] [" Iverson G." "Iverson G"] [" Fazio VC," "Fazio VC"] [" Swanik CB," "Swanik CB"] [" Broglio SP," "Broglio SP"] [" Covassin T," "Covassin T"] [" Broglio SP," "Broglio SP"] [" Chen JK," "Chen JK"] [" Van Kampen DA," "Van Kampen DA"] [" Broglio SP," "Broglio SP"] [" Pellman EJ," "Pellman EJ"] [" Pellman EJ," "Pellman EJ"] [" Schatz P," "Schatz P"] [" Biasca N," "Biasca N"] [" Collins M," "Collins M"] [" Lovell MR," "Lovell MR"] [" Lovell MR," "Lovell MR"] [" Iverson GL," "Iverson GL"] [" Cantu RC," "Cantu RC"] [" McClincy MP," "McClincy MP"] [" Schatz P," "Schatz P"] [" Iverson GL," "Iverson GL"] [" Van Kampen DA," "Van Kampen DA"] [" Lovell M," "Lovell M"] [" Mihalik JP," "Mihalik JP"] [" Moser RS," "Moser RS"] [" Broshek DK," "Broshek DK"] [" Grove R," "Grove R"] [" McCrea M," "McCrea M"] [" McCrory P," "McCrory P"] [" Iverson GL," "Iverson GL"] [" Lovell MR," "Lovell MR"] [" Bruce JM," "Bruce JM"] [" Pellman EJ," "Pellman EJ"] [" Iverson GL," "Iverson GL"] [" Lovell MR," "Lovell MR"] [" Kontos A," "Kontos A"] [" Collins MW," "Collins MW"] [" Iverson GL," "Iverson GL"] [" Lovell M," "Lovell M"] [" Field M," "Field M"] [" Covassin T," "Covassin T"] [" Iverson GL," "Iverson GL"] [" Lovell MR," "Lovell MR"] [" Collins MW," "Collins MW"] [" Lovell MR," "Lovell MR"] [" Collins MW," "Collins MW"] [" Collins MW," "Collins MW"] [" Collins MW," "Collins MW"] [" Maroon JC," "Maroon JC"] [" Lovell MR," "Lovell MR"] [" Lovell MR." "Lovell MR"] [" Aubry M," "Aubry M"] [" Grindel SH," "Grindel SH"] [" Collins MW," "Collins MW"] [" Lovell MR," "Lovell MR"] [" Collins MW," "Collins MW"] [" Lovell MR," "Lovell MR"])
于 2014-07-10T15:09:18.743 回答
1

为了说明...

注意:我将使用 html 作为前缀

(require '[net.cgrand.enlive-html :as html])

的输出(references)是一系列单独的参考元素,如

(def data-sample 
  '{:tag :li, :attrs nil, 
    :content 
    ("\n\t\t\t\t\t\t\t\t\t\t\t\t\t" 
      {:tag :strong, :attrs nil, 
       :content 
       ("AMPAR peptide values *snip*.")}
      " Dambinova SA, Shikuev, Weissman JD, Mullins, JD. "
      {:tag :em, :attrs nil, :content ("Military Medicine.")}
      " 2013, 178 (3):285-290.\t\t\t\t\t\t\t\t\t\t\t\t")})

你会注意到文章标题是粗体,期刊是斜体,所以我们至少可以使用选择器来提取它们。但是,由于格式的更改用于提供组件的视觉分离,它们也将提供数据的分离。

(defn trimmed-text-only [html-data] 
  (as-> html-data x
    (html/select x [html/text-node])
    (map clojure.string/trim x)
    (remove empty? x)))

(trimmed-text-only data-sample)
;=> 
("AMPAR peptide values *snip*." 
 "Dambinova SA, Shikuev, Weissman JD, Mullins, JD." 
 "Military Medicine." 
 "2013, 178 (3):285-290.")

这已经使组件显而易见,但请注意,每个组件都由一个句点分隔,而句点未在组件中使用。因此,我们也可以完全忽略格式并按句点分割,另外还有删除这些句点的好处。

(defn extract-major-reference-components
  [html-data]
  (as-> html-data x
    (trimmed-text-only x)
    (apply str x)
    (clojure.string/split x #"\.")
    (zipmap [:title :authors :journal :issue-ref] x)))

(extract-major-reference-components data-sample)
;=> 
{:title "AMPAR peptide values *snip*"
 :authors "Dambinova SA, Shikuev, Weissman JD, Mullins, JD",
 :journal "Military Medicine",
 :issue-ref "2013, 178 (3):285-290"}

现在您可以将此提取函数映射到引用序列上。使用输出映射,您可以使用更新输入和正则表达式进行进一步的转换,例如将单个作者或年份、问题编号和页面与问题参考分开。

于 2014-07-10T16:48:07.460 回答