1. While you are free to lurk, we welcome you to register for a (free) account so that you will be able to participate in forum discussions.

Create Web Directories From Scratch - Tutorial

Discussion in 'Directory Scripts/Themes' started by SimplySidy, Feb 9, 2015.

  1. SimplySidy Member


    Thanks to an earlier thread from our dear @Ray which made me realize that I do need to post something related to web directories. To be honest, I am not that knowledgeable when it comes to web-directories of today. I had used web directories when I was doing SEO stuff as an intern during 2007 for an organization which later put me to wordpress and other programming.

    Today, I am more of a Coder than an SEO doer. So I decided to come up with a thread which could lay down the basics for all those who want to develop Web Directories.

    Let me start by mentioning - there are lots and lots of great themes, scripts and platforms already available to do this task (of creating a web directory). And also, for now, let us also keep aside their usage etc beyond the scope of this discussion (in terms of - say, Script X does this task easily). Their mention will come in at a later stage here.

    Okay, so the question (lame, complex or whatever you feel good to call it) is -
    How do I create a web directory from scratch?

    For the sake of this discussion, assume, I already have a set of colors, a layout in mind. I do not know the ABCs of web directories except the few facts -
    1. Websites and information about them is stored on a web directory under categories/sub-categories
    2. These are good for SEO reasons
    3. These are quite large and complex websites
    4. These are used by people to search information on websites) and
    5. I only know PHP, MySQL, HTML, CSS and jQuery

    So, let us first list down the important modules that I would need to think about.

    1. My objective is to create a thread that documents the Complexities/Things that coders should be aware of when thinking about creating a web directory. I also hope that you appreciate the usefulness of such a thread here and put in your responses keeping them to the topic.

  2. Ray Active Member


    Hello SimplySidy!

    I'm going to assume here that your question is from the perspective of someone wanting to create their own web directory, where he/she would add listings and grow additional content. Not the actual "building" of the web directory website, from the technical perspective.

    The number one thing that anyone needs to have in order to run and develop a successful web directory is a working knowledge of directory editing.

    The number one factor for success or failure for directories is how well their organizational structures are built, category relevance (depending on the niche or theme), and how well their listings are presented. It took me many years of trial & error, to finally learn the ropes, instead of following the advise of others in the business.

    Their advise was very simple; "become a volunteer editor for a major directory, and get the skills you'd need to run your own directory". It would have saved me a lot of wasted time and effort in the long run.

  3. toradrake Member


    Lets say you are interested in the technicalities of building a web directory... creation and such. What are some resources you would recommend for people to use that is created for beginners?

  4. Converse Active Member


    One of the chief technicalities would be to decide which script or platform you are going to use for your directory. However, in order to do that, you need to decide what you hope to accomplish. The resource part of it, then, would be to view the features of each of the available scripts, in order to determine which will do what you need it to do. As I have learned, though, is it not always easy to determine the true value of a script from the features listed on the web site, although that's certainly a good place to start, purely because these features are advertising and, in advertising, there may be some exaggeration or an underestimation of the difficulty in accomplishing something.

    So, once you have narrowed it down to two or three choices, it's good to ask around among those who are familiar with these scripts. There too, be aware that there is a tendency for people to recommend whatever it is that they happen to be using, unless they've had a horrible experience with it. For example, someone who has been running a directory with phpLD might suggest that phpLD is the only good choice, when they may have never tried phpMyDirectory, eSyndiCat, or any of the others.

    Look at some of the directories that you like, and see what they are using. In reviewing directories, I have come across people who have insisted that they weren't using a script, that their directory was completely custom-made yet, in one case, the script was clearly identified in the code. They may have heavily modified it, but it still began with a script.

    Another wrote to correct me when, in my review, I had mentioned that it was built on a WordPress platform. Again, he insisted that it was built from the ground up. The problem was that, not only was it built on a WordPress platform, but he was using the same theme that I was using, and I recognized it. I didn't argue with him, and I suppose it is entirely possible that he had hired someone to build a directory platform from the ground up, and that person had instead modified a copy of DirectoryPress. He had done a nice job with it, and my remarks weren't intended as criticism, but it was still a WordPress theme.

    Once you've decided on a platform, you should work on your taxonomy, assuming that you've already decided on the type of directory you were going to build (general, topical, niche). The taxonomy is the system of categories and subcategories that you are going to use. Don't buy the script, theme, or whatever it is that you've decided on yet, because you may find that it won't work with your taxonomy.

    As with any other type of writing, I find it easier to avoid copying someone else's taxonomy if I don't use anyone else's directory or directories as a basis; rather, deciding for myself which organizational structure might make sense. Since I have worked with taxonomy, not only in my own directories (the one I have online and two others that are in progress), but also with other directories that have employed me, I use a mind map for that. I use Mind Node, but there are others.

    In developing taxonomy, it is very easy to get so wrapped up in the first thought that comes across my mind that, to use a cliche, I can't see the forest for the trees, so at some point, I will bog down. It took two years of wrangling for the staff of DMOZ to decide upon its regional taxonomy and, I am proud to say, the final result was very much the one that I had proposed. Heated debate was involved, as there were some who wanted to lump small towns in with the nearest larger cities, as in metropolitan areas; in the end, we decided to create metropolitan areas as separate categories, as we did counties. Others wanted to simply copy the structure of the Yahoo! Directory, arguing that it would be easier to accomplish, and that users would find it easier to use something they were already familiar with. As it turned out, Yahoo copied large parts of our taxonomy once we implemented it.

    Moving on, though, once you have a taxonomy that you think might work, wait at least a week before implementing it, since it is very difficult, and can be dangerous, in terms of SEO, to make major changes to it later. Adding to it is always fine, but changing it can break pages, and that's never good.

    During that time, you can look at some other directories and see if they have any ideas that you can modify or otherwise make use of. There will be some parts of the taxonomy that you have devised that you aren't entirely satisfied with or some other issues that you haven't been able to resolve to your satisfaction so, at this point, it helps to look at what others have done. If you find that they have been able to resolve an issue for you, at least think about maybe changing the wording a bit. Look at the ways in which different directories have built their regional categories, for example. Although they are essentially the same thing, some directories simply call that category tree "Regional," while others use "Local & Global," "By Region," "Geographical," or other wordings.

    In putting your taxonomy together, you might come up with something that would be enhanced by the ability to use @links, see also links, above or below the line capabilities, or other features. If so, find out if your intended platform will accomplish that, or how hard it might be to customize it in order to accomplish that. Are there any other features that you need, and will the platform allow you to do that? At this point, you might consider a different platform that already has that built in.

    There's more but that's a good start.

    Ray likes this.
  5. toradrake Member


    Thank you Professor @Converse that was very educational. LMAO I am still working on trying to determine what platform to work with on making my directory. Like you said, it is not easy to determine the hype from the facts out there. Can you give me a list of the choices so that I can make certain I have them all? I want to make certain that I don't leave one out that I can consider. LOL trying to work on a forum and directory is the hardest parts of my site. Everything else is a breeze.

  6. Converse Active Member


    I have a list of some of them in my directory of directories. There are others, as updating this category is on my to-do list. I know that DirectoryPress has a new product now, but I wouldn't recommend using any of the PremiumPress products for reasons that I think I've stated elsewhere in the forum. There are other WordPress-based scripts that look very good, but I keep reminding myself that DirectoryPress looked very good to me when I decided to buy it to. Of these, Vantage, in particular, looks good. It's a responsive theme and it looks good from the demo but, again, I haven't used it and cannot personally vouch for it. I am probably going to buy another directory script for a seniors directory, that I am going to run in conjunction with my seniors forum, but I believe I will be going with phpMyDirectory for that one, although I was strongly considering Vantage.

  7. toradrake Member


    I just took a look at Vantage and I liked what I saw also.... well all but the price. LOL. The phpMyDirectory does look good and much cheaper. If you do ever get around to using Vantage, you should do a thread on it to let us know if it really is worth the prices they are asking for it.

  8. Converse Active Member


    Vantage: $99 for just the directory theme.
    phpMyDirectory = $140

  9. xTinx New Member


    So how do you get all the data for your directory? The data-gathering process always baffles me because in your explanation it seems you already collected the data you needed for your directories. Are you going to scrape hundreds and thousands of websites using a crawler/bot?

  10. Converse Active Member


    For more than fifteen years now, I have been collecting data for one web directory or another. Reputable web directories have staff who do just that, moving from one category to another, locating useful sites to list in them. For the past several years, I have worked from 30-50 hours a week for one directory, paid hourly.

    Some editors may work differently, but generally several sites are brought up in tabs that may be relevant to the category topic. These sites are then visited by the editor and, if it is determined that they would add value to the category, a title and a description are written, and the site is added. That's basic. Some directories also include article-length informational articles about each of their categories, adding textual content to the outgoing directory links. 90-98% of the sites listed in the better directories were discovered and added by directory editors, who may be paid hourly or by the edit, except for a very few directories, such as DMOZ and JoeAnt, that manage to maintain a volunteer crew, which is a lot of work in itself.

    Some directory scripts include tools in which sites may be brought into the category based on keywords, even importing meta titles and meta descriptions. However, no reputable directory would leave it like that. Each site would still have to be visited, titles changed and descriptions modified to meet the directory's standards.

  11. toradrake Member


    I was going to do it manually by first listing the companies I have affiliate links with and then checking niche directories to find relevant sites for my categories. Mean while, I would have a ad up letting web masters know that they can list their sites for a low fee on my directory. It will take a lot of time and work, but I think I can do it. Just have to get the thing built first. LOL

  12. Sherell New Member


    Hi Tora...how did your directory turn out?

  13. OhioTom76 New Member


    Are we talking about building a directory completely from scratch, as in coding it from the ground up in, say, PHP, Javascript and MySQL for example? Or are we talking about obtaining an off the shelf directory script, installing it, configuring it and customizing the look and feel of it to our liking? Some of the threads on here so far sound more like the later. To code one completely from scratch seems a bit like reinventing the wheel considering all the different scripts for directories already out there on the market. It would sort of be like developing another bulletin board script - it would take years of catching up to get to the level of features that the more established ones already have and have had for some time now.

    I think one of the other challenges you might run into with a directory script, especially if there are thousands of categories and subcategories nested away in it, is how well it would function on a standard hosting plan. I know in the past when I tried to bulk add tens of thousands of pages to a Wordpress installation, it would crap out on me, and what pages that did load into it made the side run so painfully slow it wouldn't be practical to use.

  14. thzfsdhdty Active Member


    Liên hệ: cô Hồng 0124 868 55 84

    Tuyển gấp nhân viên kinh doanh sản phẩm Yến xào, dây nịt da, mỹ phẩm ...
    Lương cơ bản: thỏa thuận
    Kèm các phụ cấp: xăng, điện thoại, hoa hồng, doanh số ...
    Làm việc tại: 158/38/26 Phạm Văn Chiêu, P9, Gò Vấp, Tp.HCM
    Liên hệ: cô Hồng 0124 868 55 84

    Tuyển nhân viên làm Tết, tuyển nhân viên Kinh doanh

  15. thzfsdhdty Active Member


    Liên hệ: cô Hồng 0124 868 55 84

    Tuyển gấp nhân viên kinh doanh sản phẩm Yến xào, dây nịt da, mỹ phẩm ...
    Lương cơ bản: thỏa thuận
    Kèm các phụ cấp: xăng, điện thoại, hoa hồng, doanh số ...
    Làm việc tại: 158/38/26 Phạm Văn Chiêu, P9, Gò Vấp, Tp.HCM
    Liên hệ: cô Hồng 0124 868 55 84

    Tuyển nhân viên làm Tết, tuyển nhân viên Kinh doanh


Share This Page