Ingeniería de Sistemas · 2009
Characterization of the Colombian web
Full text 49 pages with text
Read the full thesis Record in the repository
Contents
- 1. Introductionp. 9
- 1.1. How is the Web?p. 9
- 1.2. Studying the Web of a countryp. 10
- 1.3. Collecting pagesp. 11
- 1.4. Seedsp. 11
- 1.4.1. Obtaining the seedsp. 11
- 1.5. Difficulties of the characterization of the Webp. 13
- 1.6. Structure of this Reportp. 14
- 2. Characteristics of the Web pagesp. 15
- 2.1. Downloaded pages vs invalid linksp. 15
- 2.2. Text on the pagesp. 16
- 2.3. Dynamic pagesp. 17
- 2.4. Documents that are not HTMLp. 18
- 2.5. Audio, Video and Imagesp. 18
- 2.6. Software, Source code and Compressed filesp. 19
- 2.7. Links between Web pagesp. 20
- 2.8. Ordering using link analysis algorithmsp. 21
- 3. Characteristics of the Websitesp. 24
- 3.1. Number of pagesp. 24
- 3.2. Sites with only one pagep. 25
- 3.3. Sites with many pagesp. 26
- 3.3. Size of the pages of a complete Websitep. 27
- 3.4. Agep. 29
- 3.5. Internal Linksp. 30
- 3.6. Links among Websitesp. 31
- 3.7. Most referenced Websitesp. 32
- 3.8. Sites with the most number of linksp. 33
- 3.9. Sum of the scores by linksp. 34
- 3.10. Strongly connected componentsp. 35
- 3.11. Structure of links among Websitesp. 36
- 4. Characteristics of the domainsp. 38
- 4.1. IP address and hosting providerp. 38
- 4.2. Web server softwarep. 39
- 4.3. Number of sites per Domainp. 40
- 4.4. Number of pages per domainp. 41
- 4.5. Total size of the Domainsp. 42
- 4.6. Top level Domainsp. 43
- 4.7. External Top level Domainsp. 44
- 5. Conclusionsp. 45
- 6. List of termsp. 46
- 7. Referencesp. 48
- Illustration 1: Search Only in Colombian Pagesp. 11
- Illustration 2: Google Error automated requestsp. 12
- Drawing 1: Random Networkp. 9
- Drawing 2: Free of Scale Networkp. 9
- Drawing 3: Proxy serverp. 12
- Drawing 4: HTTP Status Code Distributionp. 15
- Drawing 5: Content Distribution ( text with HTML )p. 16
- Drawing 6: Content Distribution ( text without HTML )p. 16
- Drawing 7: Distribution of links to dynamic pagesp. 17
- Drawing 8: Distribution of links to documents, excluding HTML pagesp. 18
- Drawing 9: Distribution of links to video filesp. 18
- Drawing 10: Distribution of links to audio filesp. 18
- Drawing 11: Distribution of links to imagesp. 18
- Drawing 12: Distribution of links to compressed filesp. 19
- Drawing 13: Distribution of links to source codep. 19
- Drawing 14: Distribution of links to softwarep. 19
- Drawing 15: Internal Gradep. 20
- Drawing 16: External Gradep. 20
- Drawing 17: Internal Cumulative Gradep. 20
- Drawing 18: External Cumulative Gradep. 20
- Drawing 19: Size vs External Gradep. 21
- Drawing 20: Size vs Internal Gradep. 21
- Drawing 21: Cumulative PageRankp. 22
- Drawing 22: PageRankp. 22
- Drawing 23: Hub Scorep. 22
- Drawing 24: Cumulative Hub Scorep. 22
- Drawing 25: Authority Scorep. 22
- Drawing 26: Cumulative Authority Scorep. 22
- Drawing 27: PageRank vs Authority scorep. 23
- Drawing 28: PageRank vs Hub scorep. 23
- Drawing 29: Hub score vs Authority scorep. 23
- Drawing 30: Cumulative number of documents per sitep. 24
- Drawing 31: Distribution of documents per sitep. 24
- Drawing 32: Cumulative size of the websitep. 27
- Drawing 33: Size of the websitep. 27
- Drawing 34: Age of the websitesp. 29
- Drawing 35: Distribution of the number of internal links per pagep. 30
- Drawing 36: Distribution of the number of internal linksp. 30
- Drawing 37: Internal Gradep. 31
- Drawing 38: External Gradep. 31
- Drawing 39: Internal Cumulative Gradep. 31
- Drawing 40: External Cumulative Gradep. 31
- Drawing 41: Sum of PageRankp. 34
- Drawing 42: Site cumulative of the sum of Pagerankp. 34
- Drawing 43: Sum of Hub scorep. 34
- Drawing 44: Site cumulative of the sum of Hub scorep. 34
- Drawing 45: Site cumulative of the sum of Authority scorep. 34
- Drawing 46: Sum of Authority scorep. 34
- Drawing 47: Strongly Connected Component Sizep. 36
- Drawing 48: Macroscopic structure of the Webp. 37
- Drawing 49: Distribution of the number of domains by IPp. 38
- Drawing 50: Operative Systemp. 39
- Drawing 51: Web Serverp. 39
- Drawing 52: Sites per Domainp. 41
- Drawing 53: Distribution of pages per Domainp. 41
- Drawing 54: Distribution of the size of the Domainsp. 42
Keywords
Graph theoryInternet (Computer network)MetadataSearch enginesTCP/IP (Computer network protocol)Web sitesWorl wide web (Information retrieval system)Dominios en InternetProyecto de Grado. Ingeniería de SistemasSitios Web en Colombia