Difference between revisions of "Routing and Filtering"

From DXSpider Documentation Wiki
Jump to: navigation, search
m (Route Filters)
 
(2 intermediate revisions by one user not shown)
Line 13: Line 13:
 
Initially when route filters were being tested we generated a "default" filter.  Unfortunately it quickly became apparent that this might suit the UK cluster network but didn't really fit anybody else. However using a default filter is an appropriate thing to do. How, is explained further on.
 
Initially when route filters were being tested we generated a "default" filter.  Unfortunately it quickly became apparent that this might suit the UK cluster network but didn't really fit anybody else. However using a default filter is an appropriate thing to do. How, is explained further on.
  
The first thing that you must do is determine whether you need to use route filtering �[1mat all�[22m. If you are a "normal" node with two or three partners and you arranged in an "official" non-looping tree type network, then you do not need to do route filtering and you will feel a lot better for not getting involved. If you are successfully using isolation then you also probably don't need to use route filtering.
+
The first thing that you must do is determine whether you need to use route filtering at all. If you are a "normal" node with two or three partners and you arranged in an "official" non-looping tree type network, then you do not need to do route filtering and you will feel a lot better for not getting involved. If you are successfully using isolation then you also probably don't need to use route filtering.
  
 
To put it simply, you should not mix Isolation and Route Filtering.  It will work, of sorts, but you will not get the expected results.  If you are using Isolation sucessfully at the moment, do not get involved in Route Filtering unless you have a good supply of aspirin!  Once you have started down the road of Route Filtering, do not use Isolation either.  Use one or the other, not both.
 
To put it simply, you should not mix Isolation and Route Filtering.  It will work, of sorts, but you will not get the expected results.  If you are using Isolation sucessfully at the moment, do not get involved in Route Filtering unless you have a good supply of aspirin!  Once you have started down the road of Route Filtering, do not use Isolation either.  Use one or the other, not both.
Line 23: Line 23:
 
Anyway, without further discouragement, let me start the process of explanation.
 
Anyway, without further discouragement, let me start the process of explanation.
  
 +
==The node_default filter==
  
  �[1m1.3.  The node_default filter�[0m
+
All normal systems should have a default routing filter and it should usually be set to send only the normal, unlooped, view of your "national" network. Here in the UK that means nodes from the UK and Eire, in EU it is more complex as the networks there grew up in a more intertwined way.
  
All normal systems should have a default routing filter and it should
+
The generic commands are:-
usually be set to send only the normal, unlooped, view of your
+
"national" network.  Here in the UK that means nodes from the UK and
+
Eire, in EU it is more complex as the networks there grew up in a more
+
intertwined way.
+
  
 +
reject/route node_default <filter_option>
  
The generic commands are:-
+
or
  
 +
accept/route node_default <filter_option>
  
      reject/route node_default <filter_option>
+
where filter_option is one of the following ...
  
      or
+
call <prefixes>
 +
call_dxcc <numbers>
 +
call_itu <numbers>
 +
call_zone <numbers>
 +
channel <prefixes>
 +
channel_dxcc <numbers>
 +
channel_itu <numbers>
 +
channel_zone <numbers>
  
      accept/route node_default <filter_option>
+
Please be careful if you alter this setting, it will affect ALL your links!  Remember, this is a default filter for node connections, not a per link default.
  
 +
For the default routing filter then you have two real choices: either a "national" view or the "safe" option of only your own callsign. Examples of each (for my node: GB7DJK) are:-
  
  where filter_option is one of the following ...
+
  acc/route node_default call_dxcc 61,38
 +
acc/route node_default call gb7djk
  
 +
GB7DJK uses the first of these. The DXCC countries can be obtained from the show/prefix command.
  
      call <prefixes>
+
The example filters shown control output TO all your partner nodes unless they have a specific filter applied to them (see next section).
      call_dxcc <numbers>
+
      call_itu <numbers>
+
      call_zone <numbers>
+
      channel <prefixes>
+
      channel_dxcc <numbers>
+
      channel_itu <numbers>
+
      channel_zone <numbers>
+
  
 +
It is also possible to control the incoming routing information that you are prepared to accept FROM your partner nodes. The reason this is necessary is to make sure that stuff like mail, pings and similar commands a) go down the correct links and b) don't loop around excessively. Again using GB7DJK as an example a typical default input filter would be something like:
  
  Please be careful if you alter this setting, it will affect �[4mALL�[24m your
+
  rej/route node_default input call_dxcc 61,38 and not channel_dxcc 61,38
links!  Remember, this is a �[4mdefault�[24m filter for node connections, not a
+
�[4mper�[24m �[4mlink�[24m default.
+
  
 +
What this does is accept node and user information for our national network from nodes that are in our national network, but rejects such information from anyone else. Although it doesn't explicitly say so, by implication, any other node information (not from the UK and Eire) is accepted.
  
For the default routing filter then you have two real choices: either
+
As I imagine it will take a little while to get one's head around all of this you can study the effect of any rules that you try by watching the debug output after having done:-
a "national" view or the "safe" option of only your own callsign.
+
Examples of each (for my node: GB7DJK) are:-
+
  
 +
set/debug filter
  
      acc/route node_default call_dxcc 61,38
+
After you have got tired of that, to put it back the way it was:-
      acc/route node_default call gb7djk
+
  
 +
unset/debug filter
  
GB7DJK uses the first of these. The DXCC countries can be obtained
 
from the �[4mshow/prefix�[24m command.
 
  
 +
==General route filtering==
  
  The example filters shown control �[4moutput�[24m �[1mTO �[22mall your partner nodes
+
Exactly the same rules apply for general route filtering. You would use either an accept filter or a reject filter like this ...
unless they have a specific filter applied to them (see next section).
+
  
 +
reject/route <node_call> <filter_option>
  
It is also possible to control the �[4mincoming�[24m routing information that
+
or
you are prepared to accept �[1mFROM �[22myour partner nodes. The reason this is
+
necessary is to make sure that stuff like mail, pings and similar
+
commands a) go down the correct links and b) don't loop around
+
excessively. Again using GB7DJK as an example a typical default input
+
filter would be something like:
+
  
 +
accept/route <node_call> <filter_option>
  
      rej/route node_default input call_dxcc 61,38 and not channel_dxcc 61,38
+
Here are some examples of route filters ...
  
 +
rej/route gb7djk call_dxcc 61,38 (send everything except UK+EIRE nodes)
 +
rej/route all                    (equiv to [very] restricted mode)
 +
acc/route gb7djk call_dxcc 61,38 (send only UK+EIRE nodes)
 +
acc/route gb7djk call gb7djk    (equiv to SET/ISOLATE)
  
What this does is accept node and user information for our national
+
In practice you will either be opening the default filter out for a partner by defining a specific filter for that callsign:-
network from nodes that are in our national network, but rejects such
+
information from anyone else. Although it doesn't explicitly say so,
+
by implication, any other node information (not from the UK and Eire)
+
is accepted.
+
  
 +
acc/route gb7baa all
 +
acc/route gb7baa input all
  
As I imagine it will take a little while to get one's head around all
+
or restricting it quite a lot, in fact making it very nearly like an isolated node, like this:-
of this you can study the effect of any rules that you try by watching
+
the debug output after having done:-
+
  
 +
acc/route pi4ehv-8 call gb7djk
 +
rej/route pi4ehv-8 input call_dxcc 61,38
  
      set/debug filter
+
This last example takes everything except UK and Eire from PI4EHV-8 but only sends him my local configuration (just a PC19 for GB7DJK and PC16s for my local users).
  
 +
It is possible to write much more complex rules, there are up to 10 accept/reject pairs per callsign per filter. For more information see the next section.
  
After you have got tired of that, to put it back the way it was:-
+
==General filter rules==
  
 +
Upto v1.44 it was not possible for the user to set their own filters. From v1.45 though that has all changed.  It is now possible to set filters for just about anything you wish.  If you have just updated from an older version of DXSpider you will need to update your new filters.  You do not need to do anything with your old filters, they will be renamed as you update.
  
      unset/debug filter
+
There are 3 basic commands involved in setting and manipulating filters.  These are accept, reject and clear.  First we will look generally at filtering. There are a number of things you can filter in the DXSpider system. They all use the same general mechanism.
  
 +
In general terms you can create a "reject" or an "accept" filter which can have up to 10 lines in it. You do this using, for example ...
  
  �[1m1.4General route filtering�[0m
+
  accept/spots .....
 +
  reject/spots .....
  
Exactly the same rules apply for general route filtering. You would
+
where ..... are the specific commands for that type of filter. There are filters for spots, wwv, announce, wcy and (for sysops) connects. See each different accept or reject command reference for more details.
use either an accept filter or a reject filter like this ...
+
      reject/route <node_call> <filter_option>
+
  
      or
+
There is also a command to clear out one or more lines in a filter.
 +
They are ...
  
      accept/route <node_call> <filter_option>
+
clear/spots 1
 +
clear/spots all
  
 +
There is clear/xxxx command for each type of filter, and you can check that your filters have worked by the command ...
  
  Here are some examples of route filters ...
+
  show/filter
  
 +
For now we are going to use spots for the examples, but you can apply the same principles to all types of filter.
  
      rej/route gb7djk call_dxcc 61,38 (send everything except UK+EIRE nodes)
+
==Types of filter==
      rej/route all                    (equiv to [very] restricted mode)
+
      acc/route gb7djk call_dxcc 61,38 (send only UK+EIRE nodes)
+
      acc/route gb7djk call gb7djk    (equiv to SET/ISOLATE)
+
  
 +
There are two main types of filter, accept or reject.  You can use either to achieve the result you want dependent on your own preference and which is more simple to do.  It is pointless writing 8 lines of reject filters when 1 accept filter would do the same thing!  Each filter has 10 lines (of any length) which are tried in order.  If a line matches then the action you have specified is taken (ie reject means ignore it and accept means take it)
  
In practice you will either be opening the default filter out for a
+
If you specify reject filters, then any lines that arrive that match the filter will be dumped but all else will be accepted.  If you use an accept filter, then ONLY the lines in the filter will be accepted and all else will be dumped. For example if you have a single line accept filter ...
  partner by defining a specific filter for that callsign:-
+
  
 +
accept/spots on vhf and (by_zone 14,15,16 or call_zone 14,15,16)
  
      acc/route gb7baa all
+
then you will ONLY get VHF spots from or to CQ zones 14, 15 and 16.
      acc/route gb7baa input all
+
  
 +
If you set a reject filter like this ...
  
  or restricting it quite a lot, in fact making it very nearly like an
+
  reject/spots on hf/cw
�[4misolated�[24m node, like this:-
+
  
 +
then you will get everything EXCEPT HF CW spots.  You could make this single filter even more flexible.  For example, if you are interested in IOTA and will work it even on CW even though normally you are not interested in CW, then you could say ...
  
      acc/route pi4ehv-8 call gb7djk
+
reject/spots on hf/cw and not info iota
      rej/route pi4ehv-8 input call_dxcc 61,38
+
  
 +
But in that case you might only be interested in iota and say:-
  
  This last example takes everything except UK and Eire from PI4EHV-8
+
  accept/spots not on hf/cw or info iota
but only sends him my local configuration (just a PC19 for GB7DJK and
+
PC16s for my local users).
+
  
 +
which achieves exactly the same thing. You should choose one or the other until you are comfortable with the way it works. You can mix them if you wish (actually you can have an accept AND a reject on the same line) but don't attempt this until you are sure you know what you are doing!
  
It is possible to write �[1mmuch �[22mmore complex rules, there are up to 10
+
You can arrange your filter lines into logical units, either for your own understanding or simply convenience. Here is an example ...
accept/reject pairs per callsign per filter. For more information see
+
the next section.
+
  
 +
reject/spots 1 on hf/cw
 +
reject/spots 2 on 50000/1400000 not (by_zone 14,15,16 or call_zone 14,15,16)
  
�[1m1.5.  General filter rules�[0m
+
What this does is to ignore all HF CW spots and also rejects any spots on VHF which don't either originate or spot someone in Europe.
  
Upto v1.44 it was not possible for the user to set their own filters.
+
This is an example where you would use a line number (1 and 2 in this case), if you leave the digit out, the system assumes '1'. Digits '0'-'9' are availableThis make it easier to see just what filters you have setIt also makes it more simple to remove individual filters, during a contest for example.
From v1.45 though that has all changedIt is now possible to set
+
filters for just about anything you wish.  If you have just updated
+
from an older version of DXSpider you will need to update your new
+
filtersYou do not need to do anything with your old filters, they
+
will be renamed as you update.
+
  
 +
You will notice in the above example that the second line has brackets.  Look at the line logically.  You can see there are 2 separate sections to it.  We are saying reject spots that are VHF or above APART from those in zones 14, 15 and 16 (either spotted there or originated there).  If you did not have the brackets to separate the 2 sections, then Spider would read it logically from the front and see a different expression entirely ...
  
  There are 3 basic commands involved in setting and manipulating
+
  (on 50000/1400000 and by_zone 14,15,16) or call_zone 14,15,16
filters.  These are �[4maccept�[24m, �[4mreject�[24m and �[4mclear�[24m.  First we will look
+
generally at filtering. There are a number of things you can filter in
+
the DXSpider system. They all use the same general mechanism.
+
  
 +
The simple way to remember this is, if you use OR - use brackets.  Whilst we are here CASE is not important. 'And BY_Zone' is just the same as 'and by_zone'.
  
  In general terms you can create a "reject" or an "accept" filter which
+
As mentioned earlier, setting several filters can be more flexible than simply setting one complex one. Doing it in this way means that if you want to alter your filter you can just redefine or remove one or more lines of it or one line. For example ...
can have up to 10 lines in it. You do this using, for example ...
+
  
 +
reject/spots 1 on hf/ssb
  
      accept/spots .....
+
would redefine our earlier example, or
      reject/spots .....
+
  
 +
clear/spots 1
  
where ..... are the specific commands for that type of filter. There
+
To remove all the filter lines in the spot filter ...
are filters for spots, wwv, announce, wcy and (for sysops) connects.
+
See each different accept or reject command reference for more
+
details.
+
  
  There is also a command to clear out one or more lines in a filter.
+
  clear/spots all
They are ...
+
  
 
+
==Filter options==
      clear/spots 1
+
      clear/spots all
+
 
+
 
+
There is clear/xxxx command for each type of filter.
+
 
+
 
+
and you can check that your filters have worked by the command ...
+
 
+
 
+
      show/filter
+
 
+
 
+
For now we are going to use spots for the examples, but you can apply
+
the same principles to all types of filter.
+
 
+
 
+
�[1m1.6.  Types of filter�[0m
+
 
+
There are two main types of filter, �[4maccept�[24m or �[4mreject�[24m.  You can use
+
either to achieve the result you want dependent on your own preference
+
and which is more simple to do.  It is pointless writing 8 lines of
+
reject filters when 1 accept filter would do the same thing!  Each
+
filter has 10 lines (of any length) which are tried in order.  If a
+
line matches then the action you have specified is taken (ie reject
+
means ignore it and accept means take it)
+
 
+
 
+
If you specify reject filters, then any lines that arrive that match
+
the filter will be dumped but all else will be accepted.  If you use
+
an accept filter, then ONLY the lines in the filter will be accepted
+
and all else will be dumped.  For example if you have a single line
+
�[4maccept�[24m filter ...
+
 
+
 
+
      accept/spots on vhf and (by_zone 14,15,16 or call_zone 14,15,16)
+
 
+
 
+
then you will �[4mONLY�[24m get VHF spots �[4mfrom�[24m or �[4mto�[24m CQ zones 14, 15 and 16.
+
 
+
 
+
If you set a reject filter like this ...
+
 
+
 
+
      reject/spots on hf/cw
+
 
+
 
+
Then you will get everything �[4mEXCEPT�[24m HF CW spots.  You could make this
+
single filter even more flexible.  For example, if you are interested
+
in IOTA and will work it even on CW even though normally you are not
+
interested in CW, then you could say ...
+
 
+
 
+
      reject/spots on hf/cw and not info iota
+
 
+
 
+
But in that case you might only be interested in iota and say:-
+
 
+
 
+
      accept/spots not on hf/cw or info iota
+
 
+
 
+
which achieves exactly the same thing. You should choose one or the
+
other until you are comfortable with the way it works. You can mix
+
them if you wish (actually you can have an accept AND a reject on the
+
same line) but don't attempt this until you are sure you know what you
+
are doing!
+
 
+
 
+
You can arrange your filter lines into logical units, either for your
+
own understanding or simply convenience. Here is an example ...
+
 
+
 
+
      reject/spots 1 on hf/cw
+
      reject/spots 2 on 50000/1400000 not (by_zone 14,15,16 or call_zone 14,15,16)
+
 
+
 
+
What this does is to ignore all HF CW spots and also rejects any spots
+
on VHF which don't either originate or spot someone in Europe.
+
 
+
 
+
This is an example where you would use a line number (1 and 2 in this
+
case), if you leave the digit out, the system assumes '1'. Digits
+
'0'-'9' are available.  This make it easier to see just what filters
+
you have set.  It also makes it more simple to remove individual
+
filters, during a contest for example.
+
 
+
 
+
You will notice in the above example that the second line has
+
brackets.  Look at the line logically.  You can see there are 2
+
separate sections to it.  We are saying reject spots that are VHF or
+
above �[4mAPART�[24m from those in zones 14, 15 and 16 (either spotted there or
+
originated there).  If you did not have the brackets to separate the 2
+
sections, then Spider would read it logically from the front and see a
+
different expression entirely ...
+
 
+
 
+
      (on 50000/1400000 and by_zone 14,15,16) or call_zone 14,15,16
+
 
+
 
+
The simple way to remember this is, if you use OR - use brackets.
+
Whilst we are here CASE is not important. 'And BY_Zone' is just the
+
same as 'and by_zone'.
+
 
+
As mentioned earlier, setting several filters can be more flexible
+
than simply setting one complex one.  Doing it in this way means that
+
if you want to alter your filter you can just redefine or remove one
+
or more lines of it or one line. For example ...
+
 
+
 
+
      reject/spots 1 on hf/ssb
+
 
+
 
+
would redefine our earlier example, or
+
 
+
 
+
      clear/spots 1
+
 
+
 
+
To remove all the filter lines in the spot filter ...
+
 
+
 
+
      clear/spots all
+
 
+
 
+
�[1m1.7.  Filter options�[0m
+
  
 
  You can filter in several different ways.  The options are listed in
 
  You can filter in several different ways.  The options are listed in
 
  the various helpfiles for accept, reject and filter.
 
  the various helpfiles for accept, reject and filter.
  
 +
==Default filters==
  
�[1m1.8.  Default filters�[0m
+
Sometimes all that is needed is a general rule for node connects. This can be done with a node_default filter.  This rule will always be followed, even if the link is isolated, unless another filter is set specifically.  Default rules can be set for nodes and users.  They can be set for spots, announces, WWV and WCY.  They can also be used for hops.  An example might look like this ...
 
+
Sometimes all that is needed is a general rule for node connects.
+
This can be done with a node_default filter.  This rule will always be
+
followed, even if the link is isolated, unless another filter is set
+
specifically.  Default rules can be set for nodes and users.  They can
+
be set for spots, announces, WWV and WCY.  They can also be used for
+
hops.  An example might look like this ...
+
 
+
 
+
      accept/spot node_default by_zone 14,15,16,20,33
+
      set/hops node_default spot 50
+
 
+
 
+
This filter is for spots only, you could set others for announce, WWV
+
and WCY.  This filter would work for ALL nodes unless a specific
+
filter is written to override it for a particular node.  You can also
+
set a user_default should you require.  It is important to note that
+
default filters should be considered to be "connected".  By this I
+
mean that should you override the default filter for spots, you need
+
to add a rule for the hops for spots also.
+
 
+
 
+
�[1m1.9.  Advanced filtering�[0m
+
 
+
Once you are happy with the results you get, you may like to
+
experiment.
+
 
+
 
+
The previous example that filters hf/cw spots and accepts vhf/uhf
+
spots from EU can be written with a mixed filter, for example ...
+
 
+
 
+
      rej/spot on hf/cw
+
      acc/spot on 0/30000
+
      acc/spot 2 on 50000/1400000 and (by_zone 14,15,16 or call_zone 14,15,16)
+
 
+
 
+
Note that the first filter has not been specified with a number.  This
+
will automatically be assumed to be number 1.  In this case, we have
+
said �[4mreject�[24m �[4mall�[24m �[4mHF�[24m �[4mspots�[24m �[4min�[24m �[4mthe�[24m �[4mCW�[24m �[4msection�[24m �[4mof�[24m �[4mthe�[24m �[4mbands�[24m �[4mbut�[24m �[4maccept�[24m �[4mall�[0m
+
�[4mothers�[24m �[4mat�[24m �[4mHF.�[24m  �[4mAlso�[24m �[4maccept�[24m �[4manything�[24m �[4min�[24m �[4mVHF�[24m �[4mand�[24m �[4mabove�[24m �[4mspotted�[24m �[4min�[24m �[4mor�[24m �[4mby�[0m
+
�[4moperators�[24m �[4min�[24m �[4mthe�[24m �[4mzones�[24m �[4m14,�[24m �[4m15�[24m �[4mand�[24m �[4m16�[24m.  Each filter slot actually has a
+
'reject' slot and an 'accept' slot. The reject slot is executed BEFORE
+
the accept slot.
+
 
+
 
+
It was mentioned earlier that after a reject test that doesn't match,
+
the default for following tests is 'accept', the reverse is true for
+
'accept'. In the example what happens is that the reject is executed
+
first, any non hf/cw spot is passed to the accept line, which lets
+
through everything else on HF.  The next filter line lets through just
+
VHF/UHF spots from EU.
+
 
+
 
+
�[1m1.10.  Basic hop control�[0m
+
 
+
In /spider/data you will find a file called hop_table.pl.  This is the
+
file that controls your hop count settings.  It has a set of default
+
hops on the various PC frames and also a set for each node you want to
+
alter the hops for.  You may be happy with the default settings of
+
course, but this powerful tool can help to protect and improve the
+
network.  The file will look something like this ...
+
 
+
 
+
      #
+
      # hop table construction
+
      #
+
 
+
      package DXProt;
+
 
+
      # default hopcount to use
+
      $def_hopcount = 5;
+
  
      # some variable hop counts based on message type
+
accept/spot node_default by_zone 14,15,16,20,33
      %hopcount =
+
set/hops node_default spot 50
      (
+
      11 => 10,
+
      16 => 10,
+
      17 => 10,
+
      19 => 10,
+
      21 => 10,
+
      );
+
  
 +
This filter is for spots only, you could set others for announce, WWV and WCY.  This filter would work for ALL nodes unless a specific filter is written to override it for a particular node.  You can also set a user_default should you require.  It is important to note that default filters should be considered to be "connected".  By this I mean that should you override the default filter for spots, you need to add a rule for the hops for spots also.
  
      # the per node hop control thingy
+
==Advanced filtering==
  
 +
Once you are happy with the results you get, you may like to experiment.
  
      %nodehops =
+
The previous example that filters hf/cw spots and accepts vhf/uhf spots from EU can be written with a mixed filter, for example ...
      (
+
      GB7ADX => {            11 => 8,
+
                              12 => 8,
+
                              16 => 8,
+
                              17 => 8,
+
                              19 => 8,
+
                              21 => 8,
+
                        },
+
  
      GB7UDX => {            11 => 8,
+
rej/spot on hf/cw
                              12 => 8,
+
acc/spot on 0/30000
                              16 => 8,
+
acc/spot 2 on 50000/1400000 and (by_zone 14,15,16 or call_zone 14,15,16)
                              17 => 8,
+
                              19 => 8,
+
                              21 => 8,
+
                        },
+
      GB7BAA => {
+
                              11 => 5,
+
                              12 => 8,
+
                              16 => 8,
+
                              17 => 8,
+
                              19 => 8,
+
                              21 => 8,
+
                        },
+
      );
+
  
 +
Note that the first filter has not been specified with a number.  This will automatically be assumed to be number 1.  In this case, we have said reject all HF spots in the CW section of the bands but accept all others at HF. Also accept anything in VHF and above spotted in or by operators in the zones 14,15 and 16.  Each filter slot actually has a 'reject' slot and an 'accept' slot. The reject slot is executed BEFORE the accept slot.
  
Each set of hops is contained within a pair of curly braces and
+
It was mentioned earlier that after a reject test that doesn't match, the default for following tests is 'accept', the reverse is true for 'accept'. In the example what happens is that the reject is executed first, any non hf/cw spot is passed to the accept line, which lets through everything else on HF.  The next filter line lets through just VHF/UHF spots from EU.
contains a series of PC frame types.  PC11 for example is a DX spot.
+
  The figures here are not exhaustive but should give you a good idea of
+
how the file works.
+
  
 +
==Basic hop control==
  
  SHould any of the nodecalls include an ssid, it is important to wrap
+
In /spider/data you will find a file called hop_table.pl. This is the file that controls your hop count settings.  It has a set of default hops on the various PC frames and also a set for each node you want to alter the hops for. You may be happy with the default settings of course, but this powerful tool can help to protect and improve the network.  The file will look something like this ...
  the whole call in single quotes, like this ...
+
  
 +
#
 +
# hop table construction
 +
#
  
      'DB0FHF-15' => {
+
package DXProt;
                              11 => 5,
+
# default hopcount to use
                              12 => 8,
+
$def_hopcount = 5;
                              16 => 8,
+
                              17 => 8,
+
                              19 => 8,
+
                              21 => 8,
+
                        },
+
  
 +
# some variable hop counts based on message type
 +
%hopcount =
 +
(
 +
  11 => 10,
 +
  16 => 10,
 +
  17 => 10,
 +
  19 => 10,
 +
  21 => 10,
 +
);
  
If you do not do this, you will get errors and the file will not work
 
as expected.
 
  
 +
# the per node hop control thingy
  
You can alter this file at any time, including whilst the cluster is
 
running.  If you alter the file during runtime, the command �[4mload/hops�[0m
 
will bring your changes into effect.
 
  
 +
%nodehops =
 +
(
 +
  GB7ADX => {            11 => 8,
 +
                        12 => 8,
 +
                        16 => 8,
 +
                        17 => 8,
 +
                        19 => 8,
 +
                        21 => 8,
 +
                    },
 +
GB7UDX => {            11 => 8,
 +
                        12 => 8,
 +
                        16 => 8,
 +
                        17 => 8,
 +
                        19 => 8,
 +
                        21 => 8,
 +
                    },
 +
GB7BAA => {
 +
                        11 => 5,
 +
                        12 => 8,
 +
                        16 => 8,
 +
                        17 => 8,
 +
                        19 => 8,
 +
                        21 => 8,
 +
                    },
 +
);
  
�[1m1.11.  Hop Control on Specific Nodes�[0m
 
  
You can set a callsign specific hop count for any of the standard
+
Each set of hops is contained within a pair of curly braces and contains a series of PC frame types.  PC11 for example is a DX spot. The figures here are not exhaustive but should give you a good idea of how the file works.
filter options so:-
+
  
 +
Should any of the nodecalls include an ssid, it is important to wrap the whole call in single quotes, like this ...
  
      set/hops gb7djk spot 4
+
  'DB0FHF-15' => {
      set/hops node_default route 10
+
                        11 => 5,
      set/hops gb7baa wcy 5
+
                        12 => 8,
 +
                        16 => 8,
 +
                        17 => 8,
 +
                        19 => 8,
 +
                        21 => 8,
 +
                    },
  
  
all work on their specific area of the protocol.
+
If you do not do this, you will get errors and the file will not work as expected.
  
 +
You can alter this file at any time, including whilst the cluster is running.  If you alter the file during runtime, the command load/hops will bring your changes into effect.
  
The �[4mset/hops�[24m command overrides any hops that you have set otherwise.
 
  
 +
==Hop Control on Specific Nodes==
  
You can show what hops have been set using the �[4mshow/hops�[24m command.
+
You can set a callsign specific hop count for any of the standard filter options so:-
  
 +
set/hops gb7djk spot 4
 +
set/hops node_default route 10
 +
set/hops gb7baa wcy 5
  
�[1m1.12.  Isolating networks�[0m
+
all work on their specific area of the protocol.
  
It is possible to isolate networks from each other on a "gateway" node
+
The set/hops command overrides any hops that you have set otherwise.
using the �[4mset/isolate�[24m �[4m<node_call>�[24m command.
+
  
 +
You can show what hops have been set using the show/hops command.
  
The effect of this is to partition an isolated network completely from
+
==Isolating networks==
another node connected to your node. Your node will appear on and
+
otherwise behave normally on every network to which you are connected,
+
but data from an isolated network will not cross onto any other
+
network or vice versa. However all the spot, announce and WWV traffic
+
and personal messages will still be handled locally (because you are a
+
real node on all connected networks), that is locally connected users
+
will appear on all networks and will be able to access and receive
+
information from all networks transparently.  All routed messages will
+
be sent as normal, so if a user on one network knows that you are a
+
gateway for another network, he can still still send a talk/announce
+
etc message via your node and it will be routed across.
+
  
 +
It is possible to isolate networks from each other on a "gateway" node using the set/isolate <node_call> command.
  
If you use isolate on a node connection you will continue to receive
+
The effect of this is to partition an isolated network completely from another node connected to your node. Your node will appear on and otherwise behave normally on every network to which you are connected, but data from an isolated network will not cross onto any other network or vice versa. However all the spot, announce and WWV traffic and personal messages will still be handled locally (because you are a real node on all connected networks), that is locally connected users will appear on all networks and will be able to access and receive information from all networks transparentlyAll routed messages will be sent as normal, so if a user on one network knows that you are a gateway for another network, he can still still send a talk/announce etc message via your node and it will be routed across.
all information from the isolated partner, however you will not pass
+
any information back to the isolated node.  There are times when you
+
would like to forward only spots across a link (maybe during a contest
+
for example).  To do this, isolate the node in the normal way and use
+
an �[4macc/spot�[24m �[4m>call<�[24m �[4mall�[24m filter to override the isolate.
+
  
 +
If you use isolate on a node connection you will continue to receive all information from the isolated partner, however you will not pass any information back to the isolated node.  There are times when you would like to forward only spots across a link (maybe during a contest for example).  To do this, isolate the node in the normal way and use an acc/spot call all filter to override the isolate.
  
�[1m1.13.  A DXSpider Filtering Tutorial�[0m
+
==A DXSpider Filtering Tutorial==
  
There is now an excellent primer/tutorial on filtering written by Jim
+
There is now an excellent primer/tutorial on filtering written by Jim Samuels, W3BG with an introduction by Dave Hawes N3RD that I strongly suggest you read.  You can read it here The DXSpider User Filtering Primer
Samuels, W3BG with an introduction by Dave Hawes N3RD that I strongly
+
suggest you read.  You can read it here The DXSpider User Filtering
+
Primer
+

Latest revision as of 13:13, 28 April 2012

Introduction

From DXSpider version 1.48, major changes were introduced to the way node connections are treated. This is part of an ongoing process to remove problems with loops and to enable talk and other functions to propagate across the whole of the worldwide cluster network. In fact, in a Spider network, it would be useful, perhaps even necessary to have loops. This would give real resilience to the network, meaning that if a link dropped, the information flow would simply come in and go out via a different route. Of course, we do not have a complete network of Spider nodes, there are other programs out there. Some of these do not have any protection from loops. Certainly AK1A does not handle loops well at all. It is therefore necessary to have some form of protection for these nodes.

In fact DXSpider has had a simple system for some time which is called isolation. This is similar to what in other systems such as clx, is passive mode. A more detailed explanation of isolation is given further below. This system is still available and, for simple networks, is probably all that you need.

The new functionality introduced in version 1.48 allows filtering the node and user protocol frames on a "per interface" basis. We call this route filtering. This is used instead of isolation.

What this really means is that you can control more or less completely which user and node management PC protocol frames pass to each of your partner nodes. You can also limit what comes into your node from your partners. It is even possible to control the settings that your partner node has for the routing information that it sends to you (using the 'cmd' command).

Route Filters

Initially when route filters were being tested we generated a "default" filter. Unfortunately it quickly became apparent that this might suit the UK cluster network but didn't really fit anybody else. However using a default filter is an appropriate thing to do. How, is explained further on.

The first thing that you must do is determine whether you need to use route filtering at all. If you are a "normal" node with two or three partners and you arranged in an "official" non-looping tree type network, then you do not need to do route filtering and you will feel a lot better for not getting involved. If you are successfully using isolation then you also probably don't need to use route filtering.

To put it simply, you should not mix Isolation and Route Filtering. It will work, of sorts, but you will not get the expected results. If you are using Isolation sucessfully at the moment, do not get involved in Route Filtering unless you have a good supply of aspirin! Once you have started down the road of Route Filtering, do not use Isolation either. Use one or the other, not both.

You will only require this functionality if you are "well-connected". What that means is that you are connected to several different parts of (say) the EU cluster and, at the same time, also connected to two or three places in the US which, in turn are connected back to the EU. This is called a "loop" and if you are seriously looped then you need filtering.

I should at this stage give a little bit of background on filters. All the filters in Spider work in basically the same way. You can either accept or reject various options in order to create the filter rules you wish to achieve. Some filters are user settable, others can only be altered by the sysop. Route filtering can only be done by the sysop.

Anyway, without further discouragement, let me start the process of explanation.

The node_default filter

All normal systems should have a default routing filter and it should usually be set to send only the normal, unlooped, view of your "national" network. Here in the UK that means nodes from the UK and Eire, in EU it is more complex as the networks there grew up in a more intertwined way.

The generic commands are:-

reject/route node_default <filter_option>

or

accept/route node_default <filter_option>

where filter_option is one of the following ...

call <prefixes>
call_dxcc <numbers>
call_itu <numbers>
call_zone <numbers>
channel <prefixes>
channel_dxcc <numbers>
channel_itu <numbers>
channel_zone <numbers>

Please be careful if you alter this setting, it will affect ALL your links! Remember, this is a default filter for node connections, not a per link default.

For the default routing filter then you have two real choices: either a "national" view or the "safe" option of only your own callsign. Examples of each (for my node: GB7DJK) are:-

acc/route node_default call_dxcc 61,38
acc/route node_default call gb7djk

GB7DJK uses the first of these. The DXCC countries can be obtained from the show/prefix command.

The example filters shown control output TO all your partner nodes unless they have a specific filter applied to them (see next section).

It is also possible to control the incoming routing information that you are prepared to accept FROM your partner nodes. The reason this is necessary is to make sure that stuff like mail, pings and similar commands a) go down the correct links and b) don't loop around excessively. Again using GB7DJK as an example a typical default input filter would be something like:

rej/route node_default input call_dxcc 61,38 and not channel_dxcc 61,38

What this does is accept node and user information for our national network from nodes that are in our national network, but rejects such information from anyone else. Although it doesn't explicitly say so, by implication, any other node information (not from the UK and Eire) is accepted.

As I imagine it will take a little while to get one's head around all of this you can study the effect of any rules that you try by watching the debug output after having done:-

set/debug filter

After you have got tired of that, to put it back the way it was:-

unset/debug filter


General route filtering

Exactly the same rules apply for general route filtering. You would use either an accept filter or a reject filter like this ...

reject/route <node_call> <filter_option>

or

accept/route <node_call> <filter_option>

Here are some examples of route filters ...

rej/route gb7djk call_dxcc 61,38 (send everything except UK+EIRE nodes)
rej/route all                    (equiv to [very] restricted mode)
acc/route gb7djk call_dxcc 61,38 (send only UK+EIRE nodes)
acc/route gb7djk call gb7djk     (equiv to SET/ISOLATE)

In practice you will either be opening the default filter out for a partner by defining a specific filter for that callsign:-

acc/route gb7baa all
acc/route gb7baa input all

or restricting it quite a lot, in fact making it very nearly like an isolated node, like this:-

acc/route pi4ehv-8 call gb7djk
rej/route pi4ehv-8 input call_dxcc 61,38

This last example takes everything except UK and Eire from PI4EHV-8 but only sends him my local configuration (just a PC19 for GB7DJK and PC16s for my local users).

It is possible to write much more complex rules, there are up to 10 accept/reject pairs per callsign per filter. For more information see the next section.

General filter rules

Upto v1.44 it was not possible for the user to set their own filters. From v1.45 though that has all changed. It is now possible to set filters for just about anything you wish. If you have just updated from an older version of DXSpider you will need to update your new filters. You do not need to do anything with your old filters, they will be renamed as you update.

There are 3 basic commands involved in setting and manipulating filters. These are accept, reject and clear. First we will look generally at filtering. There are a number of things you can filter in the DXSpider system. They all use the same general mechanism.

In general terms you can create a "reject" or an "accept" filter which can have up to 10 lines in it. You do this using, for example ...

accept/spots .....
reject/spots .....

where ..... are the specific commands for that type of filter. There are filters for spots, wwv, announce, wcy and (for sysops) connects. See each different accept or reject command reference for more details.

There is also a command to clear out one or more lines in a filter. They are ...

clear/spots 1
clear/spots all

There is clear/xxxx command for each type of filter, and you can check that your filters have worked by the command ...

show/filter

For now we are going to use spots for the examples, but you can apply the same principles to all types of filter.

Types of filter

There are two main types of filter, accept or reject. You can use either to achieve the result you want dependent on your own preference and which is more simple to do. It is pointless writing 8 lines of reject filters when 1 accept filter would do the same thing! Each filter has 10 lines (of any length) which are tried in order. If a line matches then the action you have specified is taken (ie reject means ignore it and accept means take it)

If you specify reject filters, then any lines that arrive that match the filter will be dumped but all else will be accepted. If you use an accept filter, then ONLY the lines in the filter will be accepted and all else will be dumped. For example if you have a single line accept filter ...

accept/spots on vhf and (by_zone 14,15,16 or call_zone 14,15,16)

then you will ONLY get VHF spots from or to CQ zones 14, 15 and 16.

If you set a reject filter like this ...

reject/spots on hf/cw

then you will get everything EXCEPT HF CW spots. You could make this single filter even more flexible. For example, if you are interested in IOTA and will work it even on CW even though normally you are not interested in CW, then you could say ...

reject/spots on hf/cw and not info iota

But in that case you might only be interested in iota and say:-

accept/spots not on hf/cw or info iota

which achieves exactly the same thing. You should choose one or the other until you are comfortable with the way it works. You can mix them if you wish (actually you can have an accept AND a reject on the same line) but don't attempt this until you are sure you know what you are doing!

You can arrange your filter lines into logical units, either for your own understanding or simply convenience. Here is an example ...

reject/spots 1 on hf/cw
reject/spots 2 on 50000/1400000 not (by_zone 14,15,16 or call_zone 14,15,16)

What this does is to ignore all HF CW spots and also rejects any spots on VHF which don't either originate or spot someone in Europe.

This is an example where you would use a line number (1 and 2 in this case), if you leave the digit out, the system assumes '1'. Digits '0'-'9' are available. This make it easier to see just what filters you have set. It also makes it more simple to remove individual filters, during a contest for example.

You will notice in the above example that the second line has brackets. Look at the line logically. You can see there are 2 separate sections to it. We are saying reject spots that are VHF or above APART from those in zones 14, 15 and 16 (either spotted there or originated there). If you did not have the brackets to separate the 2 sections, then Spider would read it logically from the front and see a different expression entirely ...

(on 50000/1400000 and by_zone 14,15,16) or call_zone 14,15,16

The simple way to remember this is, if you use OR - use brackets. Whilst we are here CASE is not important. 'And BY_Zone' is just the same as 'and by_zone'.

As mentioned earlier, setting several filters can be more flexible than simply setting one complex one. Doing it in this way means that if you want to alter your filter you can just redefine or remove one or more lines of it or one line. For example ...

reject/spots 1 on hf/ssb

would redefine our earlier example, or

clear/spots 1

To remove all the filter lines in the spot filter ...

clear/spots all

Filter options

You can filter in several different ways.  The options are listed in
the various helpfiles for accept, reject and filter.

Default filters

Sometimes all that is needed is a general rule for node connects. This can be done with a node_default filter. This rule will always be followed, even if the link is isolated, unless another filter is set specifically. Default rules can be set for nodes and users. They can be set for spots, announces, WWV and WCY. They can also be used for hops. An example might look like this ...

accept/spot node_default by_zone 14,15,16,20,33
set/hops node_default spot 50

This filter is for spots only, you could set others for announce, WWV and WCY. This filter would work for ALL nodes unless a specific filter is written to override it for a particular node. You can also set a user_default should you require. It is important to note that default filters should be considered to be "connected". By this I mean that should you override the default filter for spots, you need to add a rule for the hops for spots also.

Advanced filtering

Once you are happy with the results you get, you may like to experiment.

The previous example that filters hf/cw spots and accepts vhf/uhf spots from EU can be written with a mixed filter, for example ...

rej/spot on hf/cw
acc/spot on 0/30000
acc/spot 2 on 50000/1400000 and (by_zone 14,15,16 or call_zone 14,15,16)

Note that the first filter has not been specified with a number. This will automatically be assumed to be number 1. In this case, we have said reject all HF spots in the CW section of the bands but accept all others at HF. Also accept anything in VHF and above spotted in or by operators in the zones 14,15 and 16. Each filter slot actually has a 'reject' slot and an 'accept' slot. The reject slot is executed BEFORE the accept slot.

It was mentioned earlier that after a reject test that doesn't match, the default for following tests is 'accept', the reverse is true for 'accept'. In the example what happens is that the reject is executed first, any non hf/cw spot is passed to the accept line, which lets through everything else on HF. The next filter line lets through just VHF/UHF spots from EU.

Basic hop control

In /spider/data you will find a file called hop_table.pl. This is the file that controls your hop count settings. It has a set of default hops on the various PC frames and also a set for each node you want to alter the hops for. You may be happy with the default settings of course, but this powerful tool can help to protect and improve the network. The file will look something like this ...

#
# hop table construction
#
package DXProt;
# default hopcount to use
$def_hopcount = 5;
# some variable hop counts based on message type
%hopcount =
(
 11 => 10,
 16 => 10,
 17 => 10,
 19 => 10,
 21 => 10,
);


# the per node hop control thingy


%nodehops =
(
 GB7ADX => {            11 => 8,
                        12 => 8,
                        16 => 8,
                        17 => 8,
                        19 => 8,
                        21 => 8,
                   },
GB7UDX => {             11 => 8,
                        12 => 8,
                        16 => 8,
                        17 => 8,
                        19 => 8,
                        21 => 8,
                   },
GB7BAA => {
                        11 => 5,
                        12 => 8,
                        16 => 8,
                        17 => 8,
                        19 => 8,
                        21 => 8,
                   },
);


Each set of hops is contained within a pair of curly braces and contains a series of PC frame types. PC11 for example is a DX spot. The figures here are not exhaustive but should give you a good idea of how the file works.

Should any of the nodecalls include an ssid, it is important to wrap the whole call in single quotes, like this ...

 'DB0FHF-15' => {
                        11 => 5,
                        12 => 8,
                        16 => 8,
                        17 => 8,
                        19 => 8,
                        21 => 8,
                   },


If you do not do this, you will get errors and the file will not work as expected.

You can alter this file at any time, including whilst the cluster is running. If you alter the file during runtime, the command load/hops will bring your changes into effect.


Hop Control on Specific Nodes

You can set a callsign specific hop count for any of the standard filter options so:-

set/hops gb7djk spot 4
set/hops node_default route 10
set/hops gb7baa wcy 5

all work on their specific area of the protocol.

The set/hops command overrides any hops that you have set otherwise.

You can show what hops have been set using the show/hops command.

Isolating networks

It is possible to isolate networks from each other on a "gateway" node using the set/isolate <node_call> command.

The effect of this is to partition an isolated network completely from another node connected to your node. Your node will appear on and otherwise behave normally on every network to which you are connected, but data from an isolated network will not cross onto any other network or vice versa. However all the spot, announce and WWV traffic and personal messages will still be handled locally (because you are a real node on all connected networks), that is locally connected users will appear on all networks and will be able to access and receive information from all networks transparently. All routed messages will be sent as normal, so if a user on one network knows that you are a gateway for another network, he can still still send a talk/announce etc message via your node and it will be routed across.

If you use isolate on a node connection you will continue to receive all information from the isolated partner, however you will not pass any information back to the isolated node. There are times when you would like to forward only spots across a link (maybe during a contest for example). To do this, isolate the node in the normal way and use an acc/spot call all filter to override the isolate.

A DXSpider Filtering Tutorial

There is now an excellent primer/tutorial on filtering written by Jim Samuels, W3BG with an introduction by Dave Hawes N3RD that I strongly suggest you read. You can read it here The DXSpider User Filtering Primer