605.744: Information Retrieval, Fall 2017

Fall 2017 students -- Welcome!

Lecturer

Paul McNamee

Textbooks

• Introduction to Information Retrieval by Manning, Raghavan, and Schütze, Cambridge University Press, 2008. You are welcome to use the free, online version of the text which is available in digital form at the companion web site.

Course Times and Location

• Location: online
• Email should be the primary means of out-of-class communication; however I can meet with students in person by appointment.

Course Overview

This course covers the storage and retrieval of unstructured digital information. Topics include automatic index construction, retrieval models, textual representations, efficiency issues, search engines, text classification, and multilingual retrieval.

Grading Policy

Work for the class includes homework assignments, an independent research project, exams, and scholarly engagement. Refer to the course outline and syllabus for details. I assign grades with plus/minus modifiers (e.g., A+, B-, etc...)

Course related web-links

• Sources for on-line papers: CiteSeer ACL Anthology TREC Publications ACM Digital Library
• IR Textbooks: Information Retrieval: Implementing and Evaluating Search Engines, Managing Gigabytes, Information Retrieval: Algorithms and Heuristics Readings in Information Retrieval (Amazon), Foundations of Statistical Natural Language Processing
• IR Evaluations: TREC, CLEF, NTCIR, FIRE ROMIP (a Russian language evaluation)
• Organizations that distribute corpora: LDC, ELRA
• IR Journals: JASIST, IP&M, IR ACM Transactions on Speech and Language Processing
• IR-related conferences: SIGIR, CIKM 2014, KDD, ACL 2014 WWW-2014 CEAS 2010 (email spam) AIRWeb (web spam) ISMIR (music IR)
• Blogs Probably Irrelevant, NLPers
• On-line magazines: The Noisy Channel, Search Engine Watch, D-Lib Magazine
• Peter Norvig's tutorial on spelling correction.
• Berkeley Primer: Finding Information on the Internet
• HLT Central Repository
• John Sowa's Discrete Mathematics Primer
• Web Protocols: HTML, Z39.50 (Information Retrieval)

Software Resources

• Lucene a popular open-source search engine software (see also Solr)
• Wumpus system (Univ. Waterloo)
• GALAGO
• Lemur / Indri: a language modelling IR toolkit.
• Cornell's SMART system (predates the birth of Sergey Brin or Larry Page)
• Martin Porter's Snowball stemming tool (includes Porter Stemmer):
• Jacques Savoy's stoplists in various languages (and some stemmers too)
• Managing Gigabytes mg system
• Very nice list of NLP, IR, CL, resources (i.e. parsers, taggers) at Stanford.
• University of Michigan tool suite: Clairlib
• Trigrams-n-Text (TnT) toolkit, a visible markov model tagger written by Thorsten Brants (now of Google).
• QTag a probabilistic POS-tagger.
• On-line translators: Systran, FreeTranslation.com, Google Translate, Bing's Translator
• WordNet, a lexical database for English
• Andrew McCallum's MALLET toolkit, a Java-based API for machine learning applications using Conditional Random Fields
• Wget
• Perl LWP library (at CPAN).
• Machine Learning / Data Mining tool: WEKA
• Joachim's Support Vector Machine toolkit: SVMlight
• SVM-Multiclass, a multi-class version of SVMlight.
• Python-based set of tools for NLP tasks (parsing, POS tagging, etc...): NLTK
• Machine learning in Python: scikit
• Parsing HTML (robustly) in Python: Beautiful Soup

Cool Demos

• A 'meta' search engine: Dogpile
• A question-answering system: START
• An online joke recommendation system that demonstrates collaborative filtering: JESTER
• A faux computer science paper generator, SCIgen, from MIT
• No IR system with 3 billion queries a day is going to be perfect. Best of Google Bloopers ;-).

IR Test collections

• Reuters 21578
• The University of Glasgow has archived a set of older test collections

Several Web Search Engines

• AltaVista
• Bing
• Google (a fairly exemplary engine)
• Yahoo
• A lengthy list of alternative engines from 2007

JHU Links

• The Johns Hopkins University's Center for Language and Speech Processing. During the Spring and Fall semesters CLSP runs a weekly seminar series which meets Tuesday afternoons at 12:00 on the Homewood campus. Seminars are normally in Hackerman Hall, room B-17. You can see the schedule here.
• Human Language Technology Center of Excellence
• The Whiting school's part-time program home page.

Regional Links

• The University of Maryland's computational linguistics seminar series
• DIRE (DC-area Information Retrieval Experts) - a regional meeting of IR practitioners which meets once or twice a year.