| Highwinds Software Product Documentation: Typhoon, Twister, Tornado, Hurricane, and Cyclone | ||
|---|---|---|
| <<< Previous | Chapter 2. Cyclone Product Documentation | Next >>> |
The following sections provide detailed system configuration information.
Configuration Tutorial Provides step-by-step instructions on setting up a basic system configuration.
Configuration File Specification The "final word" on system configuration.
Advanced Configuration Examples Provides detailed examples of common advanced site configurations.
GUI Configuration Provides instructions on setting up a Web-based GUI for your customers.
Highwinds has tried to make the system very easy to configure. These simple step-by-step instructions will jump-start your site configuration.
Using the cyclone.conf-EXAMPLE file as a guide, create a cyclone.conf file in the root of the install area. In that file, create a number of "Spool" objects to hold articles. Each "Spool" object must have a SpoolName, a Path, and a KiloBytes directive. These directives specify a unique name, a location, and a size for each spool. We recommend placing a large "Spool" object on each disk in your system. For example, the following lines create a 1 GigaByte spool called "Main" that will be put in the file "/news/spool/bigSpool".
<Spool> SpoolName Main Path /news/spool/bigSpool KiloBytes 1000000 </Spool> |
Using a text editor, create a feeds.conf file in the top level of the install directory. For each machine with which this system will exchange news, create an empty "Feed" Object. (Comment lines begin with a '#' character. Blank Lines are ignored.)
<Feed> # First Machine </Feed> <Feed> # Second Machine </Feed> |
For each machine that is feeding this system news, add an IncomingHostname directive.
<Feed> # First Machine (Sends news to us) IncomingHostname first.machine.com </Feed> <Feed> # Second Machine (Doesn't send news to us) </Feed> |
For each machine to which we are sending news, add an OutgoingHostname and a Subscription directive. The Subscription directive uses wildcard characters to specify which newsgroups should be sent to this machine.
<Feed> # First Machine (Sends and Receives news to and from us) IncomingHostname first.machine.com # first.machine.com gets "Everything but the alt hierarchy" OutgoingHostname first.machine.com Subscription *, !alt.* </Feed> <Feed> # Second Machine (Receives news from us) # second.machine.com gets "Just the comp and rec hierarchies" OutgoingHostname second.machine.com Subscription comp.*, rec.* </Feed> |
You are finished! You just created a configuration file that set up a Bi-directional feed to first.machine.com and a Unidirectional feed to second.machine.com. All that is left is to run the bin/start command script to bootstrap the system. When you start Cyclone for the very first time, it will create the spools you have specified in the cyclone.conf file, do not be alarmed if this takes a significant amount of time.
The IncomingGroupFilter, IncomingSubjectFilter, SubjectFilter, and IncomingPathFilter directives, allow you to filter out inappropriate groups, subjects (such as control messages), and Usenet hosts that are known sources of inappropriate content.
Review the following examples:
<Feed> # Incoming feed from news.company.com, with NO "warez" # groups or "cancel" messages # IncomingHostName news.company.com IncomingGroupFilter *.warez* IncomingSubjectFilter cmsg?cancel* </Feed> <Feed> # Outgoing feed to news.company2.com, with NO # "MAKE MONEY FAST" articles # OutgoingHostName news.company2.com SubjectFilter *MAKE?MONEY?FAST* </Feed> <Feed> # Incoming feed from news.company3.com, with NO articles # which passed through the "pirates.com" domain # IncomingHostName news.company3com IncomingPathFilter *.pirates.com </Feed> |
By using the IncomingExpectedPathStamp on all of your incoming feeds, you can prevent your site from becoming an entry point for FAKED Usenet messages.
<Feed> # Expect "news.company.com" or "company.com" as a "Path:" Stamp # IncomingHostName news.company.com IncomingExpectedPathStamp news.company.com, company.com </Feed> |
Path: news.fakecompany.com!anotherfake.com!not-for-mail!fake |
Path: news.company.com.MISMATCH!news.fakecompany.com!anotherfake.com!not-for-mail!fake |
If you are feeding a site that has multiple news servers, you will want to specify all the news servers at that site in the Aliases directive. This will avoid the unnecessary transmission of articles to that site by treating all the news servers at that site as being "equivalent" to the host you are actually feeding.
In the following example, company.com has 3 news servers (news1.company.com, news2.company.com, and news3.company.com). We are tasked with receiving articles from news1.company.com and sending the "comp" newsgroups to news2.company.com that have "world" or "usa" distribution.
<Feed> # Take an incoming feed from news1.company.com IncomingHostName news1.company.com # Send the "comp" groups with "world" and # "usa" distribution to news2.company.com OutgoingHostName news2.company.com Subscription comp.* Distribution world, usa # Make this feed recognize all the servers at company.com Alias news1.company.com, news2.company.com, news3.company.com </Feed> |
By deploying multiple machines, you can provide a high level of redundancy, failover, and disaster recovery. By feeding each of your customers with a primary feed and one or more secondary feeds from alternate sites, you can provide uninterrupted service even in the face of catastrophic failures. Cyclone provides two ways to do this efficiently:
By using the DelayTime directive, you can "time-shift" your secondary feeds to avoid having your machines "race" each other to the customer. Because secondary feeds are "delayed", the primary feed provides the articles to the customer while secondary feeds get "Refused". Only if the primary feed becomes unreachable will secondary feed articles be accepted. Because secondary feeds are "delayed", it is a good idea to give them a much larger MaxDepth than that of the primary feed.
For even greater bandwidth efficiency, you can use the BackupHostName, BackupPortNumber, and BackupTimeInterval directives to monitor your primary feeds. Doing this will cause NO traffic from the secondary sites unless the primary site becomes unreachable.
# Primary FULL outgoing feed to news.company.com # (on host news.highwinds-software.com) <Feed> OutgoingHostName news.company.com Subscription * MaxDepth 10000 </Feed> # Example 1: On a different machine, we set up a delayed # secondary feed. We delay it by 5 minutes (300 seconds) # and double the MaxDepth <Feed> OutgoingFeedName news.company.com-backup-using-delaytime OutgoingHostName news.company.com Subscription * DelayTime 300 MaxDepth 20000 </Feed> # Example 2: On a different machine, we set up a duplicate # of the primary feed. However, we use the "Backup" directives # to tell Cyclone to only service this feed if the primary feed # (on host news.highwinds-software.com) disappears. <Feed> OutgoingFeedName news.company.com-backup-using-backuphostname OutgoingHostName news.company.com Subscription * MaxDepth 10000 # # Monitor "news.highwinds-software.com" on port 119 every 10 minutes # BackupHostName news.highwinds-software.com BackupPortNumber 119 BackupTimeInterval 600 </Feed> |
If your system is very well connected and has many very well connected peers, you may have an article offered to your system by many sites all within a split-second of each other. As always, the system will accept the article from one site (refusing it from the others), process the article and offer it to all "interested" sites. Since you know that a site that offers you an article already has the article, it makes sense to avoid offering it back to them. By carefully using the OfferedCacheName directive, you can avoid this situation. Review the following example and analysis:
# Feed to and from "news.isp.com" # <Feed> IncomingHostName news.isp.com OutgoingHostName news.isp.com OfferedCacheName isp.com </Feed> # Subscription-split feed to and from "news.company.com" # <Feed> IncomingHostName news.company.com OutgoingFeedName company-ALT OutgoingHostName news.company.com Subscription !*, alt.* OfferedCacheName company.com </Feed> <Feed> OutgoingFeedName company-NOTALT OutgoingHostName news.company.com Subscription *, !alt.* OfferedCacheName company.com </Feed> |
Using the example configuration above, assume the same article is offered to the system by news.isp.com AND news.company.com within a millisecond of each other. Further assume the article was accepted from news.isp.com and refused from news.company.com.
As always, the accepted article will be enqueued and offered to remote sites that are "interested" in the article and whose Aliases do not appear on the Path: header. However, in this case, the system will avoid offering the article to news.company.com because the OfferedCache will be consulted before the article is offered. Since the article id will be in the cache, the system will know that news.company.commust already have the article and will avoid offering it.
If your system feeds many sites, or your site configuration is being managed by more than one administrator, it can be desirable to partition the configuration across multiple files. This can be done via the #include directive.
In the following example, we show a sample feeds.conf file that uses multiple #include directives to include many configuration files. In turn, these configuration files can include other configuration files. Using this technique, the system configuration can be delegated to files under the control of different administrators.
# # TOP Level feeds.conf file # # NorthEast Region Clients #include /opt/cyclone/northeast.conf # SouthEast Region Clients #include /opt/cyclone/southeast.conf # Include our local feeds.conf file #include ../local_feeds.conf # Include the NFS mounted feeds.conf file #include /nfs/news1/cyclone.feeds.conf |
Large sites generally have a collection of interconnected News machines. By default, articles passing through any of these systems will have their Path: header line augmented with the Fully Qualified Domain Name of the local host. Since the presence of a hostname in the Path: header line affects the routing of an article, controlling what goes into that line is sometimes desirable.
Administrators have two options for controlling the way Cyclone updates the Path: header:
-masquerade: This option to cycloned lets the administrator use an argument name instead of the Fully Qualified Domain Name in the Path: header line.
-alias: This option to cycloned gives the administrator a way to add an argument name to the Path: header line if it isn't already present. This option is usually used to add a common name that represents your collection of news machines.
Contact Highwinds Software Support if you would like additional assistance or advice in using these options.
These instructions will guide you through the steps necessary to set up a Web-based GUI control page for each of your customers.
Using the #include directive, organize your Cyclone configuration file into a single tier hierarchy. That is, have a master feeds.conf file that uses the #include directive to include slave feed configuration files.
By using this simple hierarchy, you can isolate any customer's portion of the system configuration to a single configuration file.
Identify or install a CGI capable Web Server on a machine that has access to the Cyclone configuration file hierarchy created in the previous step. Ensure that the Web Server process has write permissions to the slave configuration files mentioned in the previous step. See the Advanced Section for other alternatives.
Read and follow the documentation inside the modifyfeed.conf file. These instructions will give you details on how you can control the actions of your customer's GUI.
Once this is complete, you should setup a cron(1M) entry to run the Cyclone bin/restart command script at regular intervals so that changes made through the GUI will take effect. We suggest a 12 hour interval.
The cgi-bin/modifyfeed.cgi program does not require a locally installed Cyclone system. For that reason, it can be installed on any Web Server on any machine on your network.
By exploiting this feature, you can use a central Web Server to manage all the configuration files for every Cyclone system on your network. If you do this, you can use rdist(1) or a similar system to distribute the resulting configuration files out to your Cyclone machines.
If you choose to rdist(1) your configuration files out to remote Cyclone machines, we suggest, in addition to using cron, modifying the bin/restart command script to only restart the system if the configuration files have changed. By taking the small amount of time to set this up, you can remotely control dozens of Cyclone machines from a central console.
To protect the configuration of your Cyclone system, we recommend using Web-based password protection on each of your client Web subdirectories. In addition, using SSL or similar technology will provide further protection from unauthorized access.
| <<< Previous | Home | Next >>> |
| Installation Guide | Up | Operations Guide |