• Admin Login
    View Item 
    •   Home
    • Research Institute in Information and Language Processing
    • Research Institute in Information and Language Processing
    • View Item
    •   Home
    • Research Institute in Information and Language Processing
    • Research Institute in Information and Language Processing
    • View Item
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    Browse

    All of WIRECommunitiesTitleAuthorsIssue DateSubmit DateSubjectsTypesJournalDepartmentPublisherThis CollectionTitleAuthorsIssue DateSubmit DateSubjectsTypesJournalDepartmentPublisher

    Administrators

    Admin Login

    Local Links

    AboutThe University LibraryOpen Access Publications PolicyDeposit LicenceCOREWIRE Copyright and Reuse Information

    Statistics

    Display statistics

    A Free Database of University Web Links: Data Collection Issues

    • CSV
    • RefMan
    • EndNote
    • BibTex
    • RefWorks
    Authors
    Thelwall, Mike
    Issue Date
    2003
    
    Metadata
    Show full item record
    Abstract
    This paper describes and gives access to a database of the link structures of 109 UK university and higher education college websites, as created by a specialist information science web crawler in June and July of 2001. With the increasing interest in web links by information and computer scientists this is an attempt to make available raw data for research that is not reliant upon the opaque techniques of commercial search engines. Basic tools for querying are also provided. The key issues concerning running an accurate web crawler are also discussed. Access is also given to the normally hidden crawler stop list with the aim of making the crawl process more transparent. The necessity of having such a list is discussed, with the conclusion that fully automatic crawling is not socially or empirically desirable because of the existence of database-generated areas of the web and the proliferation of the phenomenon of mirroring.This paper describes a free set of databases of the link structures of the university web sites from a selection of countries, as created by a specialist information science web crawler. With the increasing interest in web links by information and computer scientists this is an attempt to make available raw data for research that is not reliant upon the opaque techniques of commercial search engines. Basic tools for querying are also provided. The key issues concerning running an accurate web crawler are also discussed. Access is also given to the normally hidden crawler stop list with the aim of making the crawl process more transparent. The necessity of having such a list is discussed, with the conclusion that fully automatic crawling is not socially or empirically desirable because of the existence of database-generated areas of the web and the proliferation of the phenomenon of mirroring.
    Citation
    International Journal of Scientometrics, Informetrics and Bibliometrics, 2002/3, 6/7(1): Paper 2
    URI
    http://hdl.handle.net/2436/15919
    Additional Links
    http://www.cindoc.csic.es/cybermetrics/articles/v6i1p2.html
    Type
    Journal article
    Language
    en
    Description
    Metadata only. Full text available at above link.
    ISSN
    1137-5019
    Collections
    Research Institute in Information and Language Processing

    entitlement

     
    DSpace software (copyright © 2002 - 2023)  DuraSpace
    Quick Guide | Contact Us
    Open Repository is a service operated by 
    Atmire NV
     

    Export search results

    The export option will allow you to export the current search results of the entered query to a file. Different formats are available for download. To export the items, click on the button corresponding with the preferred download format.

    By default, clicking on the export buttons will result in a download of the allowed maximum amount of items.

    To select a subset of the search results, click "Selective Export" button and make a selection of the items you want to export. The amount of items that can be exported at once is similarly restricted as the full export.

    After making a selection, click one of the export format buttons. The amount of items that will be exported is indicated in the bubble next to export format.