aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
Diffstat
-rw-r--r--LICENSE278
-rw-r--r--Makefile8
-rw-r--r--README.rst57
-rw-r--r--cmd/six_monitor/main.go250
-rw-r--r--go.mod10
-rw-r--r--go.sum108
-rw-r--r--six/feed_parser.go159
-rw-r--r--six/helpers.go112
-rw-r--r--six/info_file_parser.go279
-rw-r--r--six/info_file_parser_test.go256
-rw-r--r--six/participant.go224
-rw-r--r--six/participant_builder.go113
12 files changed, 1854 insertions, 0 deletions
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 0000000..7d5393a
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,278 @@
1 GNU GENERAL PUBLIC LICENSE
2 Version 2, June 1991
3
4 Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
5 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
6 Everyone is permitted to copy and distribute verbatim copies
7 of this license document, but changing it is not allowed.
8
9 Preamble
10
11 The licenses for most software are designed to take away your
12freedom to share and change it. By contrast, the GNU General Public
13License is intended to guarantee your freedom to share and change free
14software--to make sure the software is free for all its users. This
15General Public License applies to most of the Free Software
16Foundation's software and to any other program whose authors commit to
17using it. (Some other Free Software Foundation software is covered by
18the GNU Lesser General Public License instead.) You can apply it to
19your programs, too.
20
21 When we speak of free software, we are referring to freedom, not
22price. Our General Public Licenses are designed to make sure that you
23have the freedom to distribute copies of free software (and charge for
24this service if you wish), that you receive source code or can get it
25if you want it, that you can change the software or use pieces of it
26in new free programs; and that you know you can do these things.
27
28 To protect your rights, we need to make restrictions that forbid
29anyone to deny you these rights or to ask you to surrender the rights.
30These restrictions translate to certain responsibilities for you if you
31distribute copies of the software, or if you modify it.
32
33 For example, if you distribute copies of such a program, whether
34gratis or for a fee, you must give the recipients all the rights that
35you have. You must make sure that they, too, receive or can get the
36source code. And you must show them these terms so they know their
37rights.
38
39 We protect your rights with two steps: (1) copyright the software, and
40(2) offer you this license which gives you legal permission to copy,
41distribute and/or modify the software.
42
43 Also, for each author's protection and ours, we want to make certain
44that everyone understands that there is no warranty for this free
45software. If the software is modified by someone else and passed on, we
46want its recipients to know that what they have is not the original, so
47that any problems introduced by others will not reflect on the original
48authors' reputations.
49
50 Finally, any free program is threatened constantly by software
51patents. We wish to avoid the danger that redistributors of a free
52program will individually obtain patent licenses, in effect making the
53program proprietary. To prevent this, we have made it clear that any
54patent must be licensed for everyone's free use or not licensed at all.
55
56 The precise terms and conditions for copying, distribution and
57modification follow.
58
59 GNU GENERAL PUBLIC LICENSE
60 TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
61
62 0. This License applies to any program or other work which contains
63a notice placed by the copyright holder saying it may be distributed
64under the terms of this General Public License. The "Program", below,
65refers to any such program or work, and a "work based on the Program"
66means either the Program or any derivative work under copyright law:
67that is to say, a work containing the Program or a portion of it,
68either verbatim or with modifications and/or translated into another
69language. (Hereinafter, translation is included without limitation in
70the term "modification".) Each licensee is addressed as "you".
71
72Activities other than copying, distribution and modification are not
73covered by this License; they are outside its scope. The act of
74running the Program is not restricted, and the output from the Program
75is covered only if its contents constitute a work based on the
76Program (independent of having been made by running the Program).
77Whether that is true depends on what the Program does.
78
79 1. You may copy and distribute verbatim copies of the Program's
80source code as you receive it, in any medium, provided that you
81conspicuously and appropriately publish on each copy an appropriate
82copyright notice and disclaimer of warranty; keep intact all the
83notices that refer to this License and to the absence of any warranty;
84and give any other recipients of the Program a copy of this License
85along with the Program.
86
87You may charge a fee for the physical act of transferring a copy, and
88you may at your option offer warranty protection in exchange for a fee.
89
90 2. You may modify your copy or copies of the Program or any portion
91of it, thus forming a work based on the Program, and copy and
92distribute such modifications or work under the terms of Section 1
93above, provided that you also meet all of these conditions:
94
95 a) You must cause the modified files to carry prominent notices
96 stating that you changed the files and the date of any change.
97
98 b) You must cause any work that you distribute or publish, that in
99 whole or in part contains or is derived from the Program or any
100 part thereof, to be licensed as a whole at no charge to all third
101 parties under the terms of this License.
102
103 c) If the modified program normally reads commands interactively
104 when run, you must cause it, when started running for such
105 interactive use in the most ordinary way, to print or display an
106 announcement including an appropriate copyright notice and a
107 notice that there is no warranty (or else, saying that you provide
108 a warranty) and that users may redistribute the program under
109 these conditions, and telling the user how to view a copy of this
110 License. (Exception: if the Program itself is interactive but
111 does not normally print such an announcement, your work based on
112 the Program is not required to print an announcement.)
113
114These requirements apply to the modified work as a whole. If
115identifiable sections of that work are not derived from the Program,
116and can be reasonably considered independent and separate works in
117themselves, then this License, and its terms, do not apply to those
118sections when you distribute them as separate works. But when you
119distribute the same sections as part of a whole which is a work based
120on the Program, the distribution of the whole must be on the terms of
121this License, whose permissions for other licensees extend to the
122entire whole, and thus to each and every part regardless of who wrote it.
123
124Thus, it is not the intent of this section to claim rights or contest
125your rights to work written entirely by you; rather, the intent is to
126exercise the right to control the distribution of derivative or
127collective works based on the Program.
128
129In addition, mere aggregation of another work not based on the Program
130with the Program (or with a work based on the Program) on a volume of
131a storage or distribution medium does not bring the other work under
132the scope of this License.
133
134 3. You may copy and distribute the Program (or a work based on it,
135under Section 2) in object code or executable form under the terms of
136Sections 1 and 2 above provided that you also do one of the following:
137
138 a) Accompany it with the complete corresponding machine-readable
139 source code, which must be distributed under the terms of Sections
140 1 and 2 above on a medium customarily used for software interchange; or,
141
142 b) Accompany it with a written offer, valid for at least three
143 years, to give any third party, for a charge no more than your
144 cost of physically performing source distribution, a complete
145 machine-readable copy of the corresponding source code, to be
146 distributed under the terms of Sections 1 and 2 above on a medium
147 customarily used for software interchange; or,
148
149 c) Accompany it with the information you received as to the offer
150 to distribute corresponding source code. (This alternative is
151 allowed only for noncommercial distribution and only if you
152 received the program in object code or executable form with such
153 an offer, in accord with Subsection b above.)
154
155The source code for a work means the preferred form of the work for
156making modifications to it. For an executable work, complete source
157code means all the source code for all modules it contains, plus any
158associated interface definition files, plus the scripts used to
159control compilation and installation of the executable. However, as a
160special exception, the source code distributed need not include
161anything that is normally distributed (in either source or binary
162form) with the major components (compiler, kernel, and so on) of the
163operating system on which the executable runs, unless that component
164itself accompanies the executable.
165
166If distribution of executable or object code is made by offering
167access to copy from a designated place, then offering equivalent
168access to copy the source code from the same place counts as
169distribution of the source code, even though third parties are not
170compelled to copy the source along with the object code.
171
172 4. You may not copy, modify, sublicense, or distribute the Program
173except as expressly provided under this License. Any attempt
174otherwise to copy, modify, sublicense or distribute the Program is
175void, and will automatically terminate your rights under this License.
176However, parties who have received copies, or rights, from you under
177this License will not have their licenses terminated so long as such
178parties remain in full compliance.
179
180 5. You are not required to accept this License, since you have not
181signed it. However, nothing else grants you permission to modify or
182distribute the Program or its derivative works. These actions are
183prohibited by law if you do not accept this License. Therefore, by
184modifying or distributing the Program (or any work based on the
185Program), you indicate your acceptance of this License to do so, and
186all its terms and conditions for copying, distributing or modifying
187the Program or works based on it.
188
189 6. Each time you redistribute the Program (or any work based on the
190Program), the recipient automatically receives a license from the
191original licensor to copy, distribute or modify the Program subject to
192these terms and conditions. You may not impose any further
193restrictions on the recipients' exercise of the rights granted herein.
194You are not responsible for enforcing compliance by third parties to
195this License.
196
197 7. If, as a consequence of a court judgment or allegation of patent
198infringement or for any other reason (not limited to patent issues),
199conditions are imposed on you (whether by court order, agreement or
200otherwise) that contradict the conditions of this License, they do not
201excuse you from the conditions of this License. If you cannot
202distribute so as to satisfy simultaneously your obligations under this
203License and any other pertinent obligations, then as a consequence you
204may not distribute the Program at all. For example, if a patent
205license would not permit royalty-free redistribution of the Program by
206all those who receive copies directly or indirectly through you, then
207the only way you could satisfy both it and this License would be to
208refrain entirely from distribution of the Program.
209
210If any portion of this section is held invalid or unenforceable under
211any particular circumstance, the balance of the section is intended to
212apply and the section as a whole is intended to apply in other
213circumstances.
214
215It is not the purpose of this section to induce you to infringe any
216patents or other property right claims or to contest validity of any
217such claims; this section has the sole purpose of protecting the
218integrity of the free software distribution system, which is
219implemented by public license practices. Many people have made
220generous contributions to the wide range of software distributed
221through that system in reliance on consistent application of that
222system; it is up to the author/donor to decide if he or she is willing
223to distribute software through any other system and a licensee cannot
224impose that choice.
225
226This section is intended to make thoroughly clear what is believed to
227be a consequence of the rest of this License.
228
229 8. If the distribution and/or use of the Program is restricted in
230certain countries either by patents or by copyrighted interfaces, the
231original copyright holder who places the Program under this License
232may add an explicit geographical distribution limitation excluding
233those countries, so that distribution is permitted only in or among
234countries not thus excluded. In such case, this License incorporates
235the limitation as if written in the body of this License.
236
237 9. The Free Software Foundation may publish revised and/or new versions
238of the General Public License from time to time. Such new versions will
239be similar in spirit to the present version, but may differ in detail to
240address new problems or concerns.
241
242Each version is given a distinguishing version number. If the Program
243specifies a version number of this License which applies to it and "any
244later version", you have the option of following the terms and conditions
245either of that version or of any later version published by the Free
246Software Foundation. If the Program does not specify a version number of
247this License, you may choose any version ever published by the Free Software
248Foundation.
249
250 10. If you wish to incorporate parts of the Program into other free
251programs whose distribution conditions are different, write to the author
252to ask for permission. For software which is copyrighted by the Free
253Software Foundation, write to the Free Software Foundation; we sometimes
254make exceptions for this. Our decision will be guided by the two goals
255of preserving the free status of all derivatives of our free software and
256of promoting the sharing and reuse of software generally.
257
258 NO WARRANTY
259
260 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
261FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
262OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
263PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
264OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
265MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
266TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
267PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
268REPAIR OR CORRECTION.
269
270 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
271WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
272REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
273INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
274OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
275TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
276YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
277PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
278POSSIBILITY OF SUCH DAMAGES.
diff --git a/Makefile b/Makefile
new file mode 100644
index 0000000..5bd7d42
--- /dev/null
+++ b/Makefile
@@ -0,0 +1,8 @@
1SRC_FILES = $(shell find . -name '*.go')
2
3six_status_exporter: test $(SRC_FILES)
4 go build -o $@ cmd/six_monitor/main.go
5
6.PHONY: test
7test:
8 go test ./...
diff --git a/README.rst b/README.rst
new file mode 100644
index 0000000..ff27969
--- /dev/null
+++ b/README.rst
@@ -0,0 +1,57 @@
1=========================================
2Seattle Internet Exchange Status Reporter
3=========================================
4
5This code is licensed under the GPL version 2.0 only. The code is maintained at
6`code.crute.me <https://code.crute.me/pomonaconsulting/six_monitoring>`_ and
7mirrored to `GitHub <https://github.com/pomonaconsulting/six_monitoring>`_.
8
9This is a server and library for parsing and reporting on the status of a
10participant at the `Seattle Internet Exchange <https://seattleix.net>`_. The
11server provides a Prometheus endpoint reporting the stats of a participant's
12ASN on the exchange. The stats are gathered from the `participant data`_
13exported by the exchange operators.
14
15The library can be used to process the SIX participants feed as well as to
16fetch and parse additional data that is exported by the SIX route servers. It's
17mainly useful for generating reports in the case of errors or building
18monitoring infrastructure around SIX data.
19
20Installing
21==========
22The default ``make`` target will create the correct binary. You can also build
23using regular ``go build``, look at the ``Makefile`` for more details.
24
25::
26
27 make
28 ./six_status_exporter YOUR-ASN
29
30By default the server will run on port ``9119`` and export a Prometheus metrics
31endpoint at ``/metrics``.
32
33Library
34=======
35The main models are located in `six/participant.go`_ and contain the important
36details about the library. To generate the models from the SIX CSV consume the
37``ParseSIXCSV`` and ``FetchParseSIXCSV`` which take a ``io.Reader`` and no
38arguments, respectively. Those functions are defined in `six/feed_parser.go`_.
39
40Contributing
41============
42The authors welcome and appreciate contributions. To contribute please open a
43pull request on the GitHub page linked above, or email a patch in ``git am``
44format to the author. To have the best contribution experience, please:
45
46* Don't break backwards compatibility of public interfaces
47* Write tests for your new feature/bug fix (run ``make test``)
48* Ensure that existing tests pass
49* Update the readme/documentation, if necessary
50* Ensure your code follows ``go fmt`` standards
51
52All code is reviewed before acceptance and changes may be requested to better
53follow the conventions of the existing API.
54
55.. _six/participant.go: https://code.crute.me/pomonaconsulting/six_monitoring/tree/six/participant.go
56.. _six/feed_parser.go: https://code.crute.me/pomonaconsulting/six_monitoring/tree/six/feed_parser.go
57.. _participant data: https://www.seattleix.net/participants/>
diff --git a/cmd/six_monitor/main.go b/cmd/six_monitor/main.go
new file mode 100644
index 0000000..cd41d8f
--- /dev/null
+++ b/cmd/six_monitor/main.go
@@ -0,0 +1,250 @@
1// SPDX-License-Identifier: GPL-2.0-only
2// Copyright (C) 2020 Michael Crute <mike@crute.us>. All rights reserved.
3//
4// Use of this source code is governed by a license that can be found in the
5// LICENSE file.
6
7package main
8
9import (
10 "flag"
11 "fmt"
12 "log"
13 "net/http"
14 _ "net/http/pprof"
15 "os"
16 "strconv"
17
18 "github.com/prometheus/client_golang/prometheus"
19 "github.com/prometheus/client_golang/prometheus/promhttp"
20 "github.com/prometheus/common/version"
21
22 "code.crute.me/pomonaconsulting/six_monitoring/six"
23)
24
25const (
26 namespace = "sixstatus"
27 exporter = "six_status_exporter"
28)
29
30var (
31 up = prometheus.NewDesc(
32 prometheus.BuildFQName(namespace, "", "up"),
33 "Was the SIX query successful?",
34 nil, nil,
35 )
36 rsPrefixCount = prometheus.NewDesc(
37 prometheus.BuildFQName(namespace, "route_server", "prefix_count"),
38 "Number of prefixes advertised to a route server.",
39 []string{"ipversion", "mtu", "server"}, nil,
40 )
41 rsErrorCount = prometheus.NewDesc(
42 prometheus.BuildFQName(namespace, "route_server", "error_count"),
43 "Number of errors reported by a route server.",
44 []string{"ipversion", "mtu", "server"}, nil,
45 )
46 rsTransitErrorCount = prometheus.NewDesc(
47 prometheus.BuildFQName(namespace, "route_server", "transit_error_count"),
48 "Number of transit errors reported by a route server.",
49 []string{"ipversion", "mtu", "server"}, nil,
50 )
51 irrPrefixCount = prometheus.NewDesc(
52 prometheus.BuildFQName(namespace, "irr", "prefix_count"),
53 "Number of prefixes resolved from IRR records.",
54 []string{"ipversion"}, nil,
55 )
56 irrASNCount = prometheus.NewDesc(
57 prometheus.BuildFQName(namespace, "irr", "asn_count"),
58 "Number of ASNs resolved from IRR records.",
59 []string{"ipversion"}, nil,
60 )
61 irrAsSetPrefixCount = prometheus.NewDesc(
62 prometheus.BuildFQName(namespace, "irr", "as_set_prefix_count"),
63 "Number of prefixes in the AS-SET resolved from IRR records.",
64 []string{"ipversion"}, nil,
65 )
66 pdbPrefixCount = prometheus.NewDesc(
67 prometheus.BuildFQName(namespace, "peeringdb", "prefix_count"),
68 "Number of prefixes configured in PeeringDB.",
69 []string{"ipversion"}, nil,
70 )
71 roaCount = prometheus.NewDesc(
72 prometheus.BuildFQName(namespace, "", "roa_count"),
73 "Number of resolved ROA records.",
74 nil, nil,
75 )
76)
77
78type SIXCollector struct {
79 ASN int
80}
81
82var _ prometheus.Collector = (*SIXCollector)(nil)
83
84func (c *SIXCollector) Describe(ch chan<- *prometheus.Desc) {
85 ch <- up
86 ch <- rsPrefixCount
87 ch <- rsErrorCount
88 ch <- rsTransitErrorCount
89 ch <- irrPrefixCount
90 ch <- irrASNCount
91 ch <- irrAsSetPrefixCount
92 ch <- pdbPrefixCount
93 ch <- roaCount
94}
95
96func routeServerMetrics(rs *six.RouteServer, ch chan<- prometheus.Metric) {
97 ch <- prometheus.MustNewConstMetric(
98 rsPrefixCount, prometheus.GaugeValue, float64(rs.IPv4.Prefixes),
99 "4", "1500", string(strconv.Itoa(rs.Number)),
100 )
101 ch <- prometheus.MustNewConstMetric(
102 rsErrorCount, prometheus.GaugeValue, float64(rs.IPv4.Errors),
103 "4", "1500", string(strconv.Itoa(rs.Number)),
104 )
105 ch <- prometheus.MustNewConstMetric(
106 rsTransitErrorCount, prometheus.GaugeValue, float64(rs.IPv4.TransitErrors),
107 "4", "1500", string(strconv.Itoa(rs.Number)),
108 )
109 ch <- prometheus.MustNewConstMetric(
110 rsPrefixCount, prometheus.GaugeValue, float64(rs.IPv6.Prefixes),
111 "6", "1500", string(strconv.Itoa(rs.Number)),
112 )
113 ch <- prometheus.MustNewConstMetric(
114 rsErrorCount, prometheus.GaugeValue, float64(rs.IPv6.Errors),
115 "6", "1500", string(strconv.Itoa(rs.Number)),
116 )
117 ch <- prometheus.MustNewConstMetric(
118 rsTransitErrorCount, prometheus.GaugeValue, float64(rs.IPv6.TransitErrors),
119 "6", "1500", string(strconv.Itoa(rs.Number)),
120 )
121 ch <- prometheus.MustNewConstMetric(
122 rsPrefixCount, prometheus.GaugeValue, float64(rs.IPv4Jumbo.Prefixes),
123 "4", "9000", string(strconv.Itoa(rs.Number)),
124 )
125 ch <- prometheus.MustNewConstMetric(
126 rsErrorCount, prometheus.GaugeValue, float64(rs.IPv4Jumbo.Errors),
127 "4", "9000", string(strconv.Itoa(rs.Number)),
128 )
129 ch <- prometheus.MustNewConstMetric(
130 rsTransitErrorCount, prometheus.GaugeValue, float64(rs.IPv4Jumbo.TransitErrors),
131 "4", "9000", string(strconv.Itoa(rs.Number)),
132 )
133 ch <- prometheus.MustNewConstMetric(
134 rsPrefixCount, prometheus.GaugeValue, float64(rs.IPv6Jumbo.Prefixes),
135 "6", "9000", string(strconv.Itoa(rs.Number)),
136 )
137 ch <- prometheus.MustNewConstMetric(
138 rsErrorCount, prometheus.GaugeValue, float64(rs.IPv6Jumbo.Errors),
139 "6", "9000", string(strconv.Itoa(rs.Number)),
140 )
141 ch <- prometheus.MustNewConstMetric(
142 rsTransitErrorCount, prometheus.GaugeValue, float64(rs.IPv6Jumbo.TransitErrors),
143 "6", "9000", string(strconv.Itoa(rs.Number)),
144 )
145}
146
147func (c *SIXCollector) Collect(ch chan<- prometheus.Metric) {
148 sp, err := six.FetchParseSIXCSV()
149 if err != nil {
150 log.Printf("error: Fetching and parsing CSV: %s", err)
151 ch <- prometheus.MustNewConstMetric(up, prometheus.GaugeValue, 0)
152 return
153 }
154
155 ps, ok := sp.GetParticipantByASN(c.ASN)
156 if !ok {
157 log.Printf("error: No participant for ASN: %d", c.ASN)
158 ch <- prometheus.MustNewConstMetric(up, prometheus.GaugeValue, 0)
159 return
160 }
161
162 p := ps[0]
163
164 ch <- prometheus.MustNewConstMetric(up, prometheus.GaugeValue, 1)
165
166 // ipversion, mtu, server
167
168 if p.RouteServer2 != nil {
169 routeServerMetrics(p.RouteServer2, ch)
170 }
171
172 if p.RouteServer3 != nil {
173 routeServerMetrics(p.RouteServer3, ch)
174 }
175
176 ch <- prometheus.MustNewConstMetric(
177 irrPrefixCount, prometheus.GaugeValue, float64(p.IRRv4.PrefixCount),
178 "4",
179 )
180 ch <- prometheus.MustNewConstMetric(
181 irrASNCount, prometheus.GaugeValue, float64(p.IRRv4.ASNCount),
182 "4",
183 )
184 ch <- prometheus.MustNewConstMetric(
185 irrAsSetPrefixCount, prometheus.GaugeValue, float64(p.IRRv4.ASSetCount),
186 "4",
187 )
188 ch <- prometheus.MustNewConstMetric(
189 pdbPrefixCount, prometheus.GaugeValue, float64(p.PeeringDBPrefixCountv4),
190 "4",
191 )
192 ch <- prometheus.MustNewConstMetric(
193 irrPrefixCount, prometheus.GaugeValue, float64(p.IRRv6.PrefixCount),
194 "6",
195 )
196 ch <- prometheus.MustNewConstMetric(
197 irrASNCount, prometheus.GaugeValue, float64(p.IRRv6.ASNCount),
198 "6",
199 )
200 ch <- prometheus.MustNewConstMetric(
201 irrAsSetPrefixCount, prometheus.GaugeValue, float64(p.IRRv6.ASSetCount),
202 "6",
203 )
204 ch <- prometheus.MustNewConstMetric(
205 pdbPrefixCount, prometheus.GaugeValue, float64(p.PeeringDBPrefixCountv6),
206 "6",
207 )
208
209 ch <- prometheus.MustNewConstMetric(roaCount, prometheus.GaugeValue, float64(p.ROACount))
210}
211
212func main() {
213 var (
214 showVersion = flag.Bool("version", false, "Print version information.")
215 listenAddress = flag.String("web.listen-address", ":9119", "Address to listen on for web interface and telemetry.")
216 )
217 flag.Parse()
218
219 if len(os.Args) != 2 {
220 fmt.Fprintf(os.Stdout, "usage: %s <ASN>\n", os.Args[0])
221 os.Exit(1)
222 }
223
224 asn, err := strconv.Atoi(os.Args[1])
225 if err != nil {
226 fmt.Fprintln(os.Stdout, "invalid ASN, must be numeric")
227 fmt.Fprintf(os.Stdout, "usage: %s <ASN>\n", os.Args[0])
228 os.Exit(1)
229 }
230
231 if *showVersion {
232 fmt.Fprintln(os.Stdout, version.Print(exporter))
233 os.Exit(0)
234 }
235 log.Println("Starting", exporter, version.Info())
236 log.Println("Build context", version.BuildContext())
237
238 prometheus.MustRegister(version.NewCollector(exporter), &SIXCollector{asn})
239
240 log.Println("Starting Server: ", *listenAddress)
241 http.Handle("/metrics", promhttp.Handler())
242 http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
243 w.Write([]byte(`<html>
244 <head><title>SIX Status Exporter</title></head><body>
245 <h1>SIX Status Exporter</h1>
246 <p><a href="/metrics">Metrics</a></p>
247 </body></html>`))
248 })
249 log.Fatal(http.ListenAndServe(*listenAddress, nil))
250}
diff --git a/go.mod b/go.mod
new file mode 100644
index 0000000..2614df2
--- /dev/null
+++ b/go.mod
@@ -0,0 +1,10 @@
1module code.crute.me/pomonaconsulting/six_monitoring
2
3go 1.13
4
5require (
6 github.com/prometheus-community/bind_exporter v0.3.0
7 github.com/prometheus/client_golang v1.4.1
8 github.com/prometheus/common v0.9.1
9 github.com/stretchr/testify v1.4.0
10)
diff --git a/go.sum b/go.sum
new file mode 100644
index 0000000..5e4f5a3
--- /dev/null
+++ b/go.sum
@@ -0,0 +1,108 @@
1github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
2github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751 h1:JYp7IbQjafoB+tBA3gMyHYHrpOtNuDiK/uB5uXxq5wM=
3github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
4github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
5github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
6github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d h1:UQZhZ2O0vMHr2cI+DC1Mbh0TJxzA3RcLoMsFw+aXw7E=
7github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho=
8github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
9github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
10github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
11github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
12github.com/cespare/xxhash/v2 v2.1.1 h1:6MnRN8NT7+YBpUIWxHtefFZOKTAPgGjpQSxqLNn0+qY=
13github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
14github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
15github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
16github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
17github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
18github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
19github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
20github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
21github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
22github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
23github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
24github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
25github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
26github.com/golang/protobuf v1.3.2 h1:6nsPYzhq5kReh6QImI3k5qWzO4PEbvbIW2cwSfR/6xs=
27github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
28github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
29github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
30github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
31github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
32github.com/json-iterator/go v1.1.8/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
33github.com/json-iterator/go v1.1.9/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
34github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
35github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
36github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
37github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
38github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
39github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
40github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
41github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU=
42github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
43github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
44github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
45github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
46github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
47github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
48github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
49github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
50github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
51github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
52github.com/prometheus-community/bind_exporter v0.3.0 h1:DsyFrvT607dHUBbHOh+h8jqNeMGP6+WKZRlZNWNMmpg=
53github.com/prometheus-community/bind_exporter v0.3.0/go.mod h1:VNrkjpy+wDFpgW+A/8+CQgMM0fISAfpSdhnat/K6+Ic=
54github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
55github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo=
56github.com/prometheus/client_golang v1.3.0/go.mod h1:hJaj2vgQTGQmVCsAACORcieXFeDPbaTKGT+JTgUa3og=
57github.com/prometheus/client_golang v1.4.1 h1:FFSuS004yOQEtDdTq+TAOLP5xUq63KqAFYyOi8zA+Y8=
58github.com/prometheus/client_golang v1.4.1/go.mod h1:e9GMxYsXl05ICDXkRhurwBS4Q3OK1iX/F2sw+iXX5zU=
59github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
60github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
61github.com/prometheus/client_model v0.1.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
62github.com/prometheus/client_model v0.2.0 h1:uq5h0d+GuxiXLJLNABMgp2qUWDPiLvgCzz2dUR+/W/M=
63github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
64github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
65github.com/prometheus/common v0.7.0/go.mod h1:DjGbpBbp5NYNiECxcL/VnbXCCaQpKd3tt26CguLLsqA=
66github.com/prometheus/common v0.9.1 h1:KOMtN28tlbam3/7ZKEYKHhKoJZYYj3gMH4uc62x7X7U=
67github.com/prometheus/common v0.9.1/go.mod h1:yhUN8i9wzaXS3w1O07YhxHEBxD+W35wd8bs7vj7HSQ4=
68github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
69github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
70github.com/prometheus/procfs v0.0.8 h1:+fpWZdT24pJBiqJdAwYBjPSk+5YmQzYNPYzQsdzLkt8=
71github.com/prometheus/procfs v0.0.8/go.mod h1:7Qr8sr6344vo1JqZ6HhLceV9o3AJ1Ff+GxbHq6oeK9A=
72github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
73github.com/sirupsen/logrus v1.4.2 h1:SPIRibHv4MatM3XXNO2BJeFLZwZ2LvZgfQ5+UNI2im4=
74github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
75github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
76github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
77github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
78github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
79github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
80github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
81golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
82golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
83golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
84golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
85golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
86golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
87golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
88golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
89golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
90golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
91golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
92golang.org/x/sys v0.0.0-20191220142924-d4481acd189f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
93golang.org/x/sys v0.0.0-20200103143344-a1369afcdac7/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
94golang.org/x/sys v0.0.0-20200122134326-e047566fdf82 h1:ywK/j/KkyTHcdyYSZNXGjMwgmDSfjglYZ3vStQ/gSCU=
95golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
96golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
97golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
98gopkg.in/alecthomas/kingpin.v2 v2.2.6 h1:jMFz6MfLP0/4fUyZle81rXUoxOBFi19VUFKVDOQfozc=
99gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
100gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
101gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
102gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
103gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
104gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
105gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
106gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
107gopkg.in/yaml.v2 v2.2.5 h1:ymVxjfMaHvXD8RqPRmzHHsB3VvucivSkIAvJFDI5O3c=
108gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
diff --git a/six/feed_parser.go b/six/feed_parser.go
new file mode 100644
index 0000000..fb839f7
--- /dev/null
+++ b/six/feed_parser.go
@@ -0,0 +1,159 @@
1// SPDX-License-Identifier: GPL-2.0-only
2// Copyright (C) 2020 Michael Crute <mike@crute.us>. All rights reserved.
3//
4// Use of this source code is governed by a license that can be found in the
5// LICENSE file.
6
7package six
8
9import (
10 "encoding/csv"
11 "fmt"
12 "io"
13 "net/http"
14 "os"
15)
16
17const sixFeedUrl = "https://www.seattleix.net/autogen/participants.csv"
18
19// Error indicating that a line in the SIX CSV feed could not be parsed.
20// Contains the actual error as well as the line number on which the error
21// occurred.
22type SIXParticipantParseError struct {
23 Line int
24 Err error
25}
26
27// Return the error string for the error
28func (e *SIXParticipantParseError) Error() string {
29 return fmt.Sprintf("record on line %d: %s", e.Line, e.Err)
30}
31
32// Returns the wrapped error
33func (e *SIXParticipantParseError) Unwrap() error {
34 return e.Err
35}
36
37// Parser for the SIX participant CSV feed.
38type SIXParser struct {
39 Records []*SIXParticipant
40 Errors []error
41 asnIndex map[int][]*SIXParticipant
42 headers []string
43}
44
45// Builds a new SIXParser
46func NewSIXParser() *SIXParser {
47 return &SIXParser{
48 Records: []*SIXParticipant{},
49 Errors: []error{},
50 asnIndex: map[int][]*SIXParticipant{},
51 headers: nil,
52 }
53}
54
55// After parsing the feed will return a list of participant structs for the
56// passed ASN as well as a boolean indicating if the participant exists.
57// Participants may have several connections to the exchange and each
58// connection is a different participant.
59func (p *SIXParser) GetParticipantByASN(asn int) ([]*SIXParticipant, bool) {
60 r, ok := p.asnIndex[asn]
61 return r, ok
62}
63
64// Adds a participant to the SIXParticipant struct
65func (p *SIXParser) AddParticipant(sp *SIXParticipant) {
66 p.Records = append(p.Records, sp)
67
68 if _, ok := p.asnIndex[sp.ASN]; !ok {
69 p.asnIndex[sp.ASN] = []*SIXParticipant{sp}
70 } else {
71 p.asnIndex[sp.ASN] = append(p.asnIndex[sp.ASN], sp)
72 }
73}
74
75func (p *SIXParser) sliceToHeaderMap(s []string) map[string]string {
76 r := map[string]string{}
77 for i, k := range p.headers {
78 r[k] = s[i]
79 }
80 return r
81}
82
83func (p *SIXParser) addRow(rn int, r []string) {
84 defer func() {
85 if e := recover(); e != nil {
86 p.Errors = append(p.Errors, &SIXParticipantParseError{
87 Line: rn,
88 Err: e.(error),
89 })
90 }
91 }()
92 p.AddParticipant(NewSIXParticipantFromData(p.sliceToHeaderMap(r)))
93}
94
95// ASSUMPTION: The correct row size will not change
96//
97// Participants that aren't using the route servers tend to have inconsistent
98// row data. The head and tail of the row are fine and contain the organization
99// and connection information but the middle area that contains route server
100// stats generally has 7-9 too few columns. This function will graft the head
101// and tail onto an appropriately sized row.
102func (p *SIXParser) fixupRow(r []string) []string {
103 k := make([]string, 49)
104 copy(k[:18], r[:18])
105 copy(k[42:], r[len(r)-7:])
106 return k
107}
108
109// Parse an io.Reader containing SIX participant data in CSV format and collect
110// the results into the parser for later querying.
111func (p *SIXParser) Parse(fr io.Reader) {
112 rn := 0
113 cr := csv.NewReader(fr)
114 for {
115 rn++
116 row, err := cr.Read()
117 if err == io.EOF {
118 return
119 }
120 if err != nil {
121 row = p.fixupRow(row)
122 p.Errors = append(p.Errors, err)
123 }
124 if p.headers == nil {
125 p.headers = row
126 continue
127 }
128 p.addRow(rn, row)
129 }
130}
131
132// Create a new SIXParser and parse the CSV file pointed to by the filename.
133func ParseSIXCSV(filename string) (*SIXParser, error) {
134 fp, err := os.Open(filename)
135 if err != nil {
136 return nil, err
137 }
138 defer fp.Close()
139
140 sp := NewSIXParser()
141 sp.Parse(fp)
142
143 return sp, nil
144}
145
146// Create a new SIXParser and parse the contents of the SIX participant file
147// locate on the SIX http server.
148func FetchParseSIXCSV() (*SIXParser, error) {
149 res, err := http.Get(sixFeedUrl)
150 if err != nil {
151 return nil, err
152 }
153 defer res.Body.Close()
154
155 sp := NewSIXParser()
156 sp.Parse(res.Body)
157
158 return sp, nil
159}
diff --git a/six/helpers.go b/six/helpers.go
new file mode 100644
index 0000000..694a79e
--- /dev/null
+++ b/six/helpers.go
@@ -0,0 +1,112 @@
1// SPDX-License-Identifier: GPL-2.0-only
2// Copyright (C) 2020 Michael Crute <mike@crute.us>. All rights reserved.
3//
4// Use of this source code is governed by a license that can be found in the
5// LICENSE file.
6
7package six
8
9import (
10 "net"
11 "strconv"
12 "strings"
13 "time"
14)
15
16func mustParseInt(a string) int {
17 a = strings.TrimSpace(a)
18
19 if a == "" {
20 return 0
21 }
22
23 i, err := strconv.Atoi(a)
24 if err != nil {
25 panic(err)
26 }
27
28 return i
29}
30
31func parseYesNo(b string) bool {
32 return strings.ToLower(strings.TrimSpace(b)) == "yes"
33}
34
35func parseIP(i string) *net.IP {
36 o := net.ParseIP(i)
37 return &o
38}
39
40func parseIPNetFromCIDR(i string) *net.IPNet {
41 _, ipnet, _ := net.ParseCIDR(strings.TrimSpace(i))
42 return ipnet
43}
44
45func mustParseTime(t string) *time.Time {
46 t = strings.TrimSpace(t)
47
48 if t == "" {
49 return nil
50 }
51
52 i, err := time.Parse("2006-01-02", t)
53 if err != nil {
54 panic(err)
55 }
56
57 return &i
58}
59
60func mustParseLongTime(t string) *time.Time {
61 t = strings.TrimSpace(t)
62
63 if t == "" {
64 return nil
65 }
66
67 i, err := time.Parse("2006-01-02 15:04:05 MST", t)
68 if err != nil {
69 panic(err)
70 }
71
72 return &i
73}
74
75func mustParseLongTimeNoZone(t string) *time.Time {
76 t = strings.TrimSpace(t)
77
78 if t == "" {
79 return nil
80 }
81
82 i, err := time.Parse("2006-01-02 15:04:05", t)
83 if err != nil {
84 panic(err)
85 }
86
87 return &i
88}
89
90func allEmpty(d []string) bool {
91 for _, v := range d {
92 if strings.TrimSpace(v) != "" {
93 return false
94 }
95 }
96 return true
97}
98
99func parseASPath(p string) []int {
100 out := []int{}
101
102 for _, i := range strings.Split(p, " ") {
103 ii, err := strconv.Atoi(strings.TrimSpace(i))
104 // Some AS paths contain { and } which are not valid and need to be discard
105 if err != nil {
106 continue
107 }
108 out = append(out, ii)
109 }
110
111 return out
112}
diff --git a/six/info_file_parser.go b/six/info_file_parser.go
new file mode 100644
index 0000000..cf98280
--- /dev/null
+++ b/six/info_file_parser.go
@@ -0,0 +1,279 @@
1// SPDX-License-Identifier: GPL-2.0-only
2// Copyright (C) 2020 Michael Crute <mike@crute.us>. All rights reserved.
3//
4// Use of this source code is governed by a license that can be found in the
5// LICENSE file.
6
7package six
8
9import (
10 "fmt"
11 "io/ioutil"
12 "net"
13 "net/http"
14 "regexp"
15 "strings"
16)
17
18const (
19 roaUrl = "https://www.seattleix.net/rs/rpki_roas/%d.txt" // ASN
20 pfxUrl = "https://www.seattleix.net/rs/irr_prefixes/%d.v%d.txt" // ASN, IP Version
21 asnUrl = "https://www.seattleix.net/rs/irr_asns/%d.v%d.txt" // ASN, IP Version
22 asSetUrl = "https://www.seattleix.net/rs/irr_as-set_prefixes/%d.v%d.txt" // ASN, IP Version
23 rsPrefixUrl = "https://www.seattleix.net/rs/rs%d.%d.v%d/%d:1:%d:v%d.txt" // RS#, MTU, IP Version, ASN, MTU, IP Version
24 rsErrUrl = "https://www.seattleix.net/rs/rs%d.%d.v%d/%d:1:%d:v%d.%serr.txt" // RS#, MTU, IP Version, ASN, MTU, IP Version, x for xerr
25)
26
27// Date, ASN, MTU, IP Version, Net Name, Network, Source, AS Path (may have {}), Error Message
28// See tests for line examples
29var errorLineRegex = regexp.MustCompile(`(?P<datetime>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) <[^>]+> (?P<asn>\d+):1:(?P<mtu>\d+):v(?P<ip_version>[46]):(?P<net_name>[^:]+): (?P<target>(?:[0-9\.\/]+|[0-9a-f\/:]+)) from (?P<source>(?:[0-9\.]+|[0-9a-f:]+?)):? (?:(?:bgp_path)?\(path (?P<path>[\d {}]+)\):?)? ?(?P<error>.*)`)
30
31func fetchParseRSErrors(rsNum, asn int, forTransit bool) *Errors {
32 transit := ""
33 if forTransit {
34 transit = "x"
35 }
36
37 // Not everyone will have every file and parseErrorFiles ignores nil so
38 // just ignore errors here
39 v4, _ := fetchFile(rsErrUrl, rsNum, 1500, IPv4, asn, 1500, IPv4, transit)
40 v6, _ := fetchFile(rsErrUrl, rsNum, 1500, IPv6, asn, 1500, IPv6, transit)
41 v4j, _ := fetchFile(rsErrUrl, rsNum, 9000, IPv4, asn, 9000, IPv4, transit)
42 v6j, _ := fetchFile(rsErrUrl, rsNum, 9000, IPv6, asn, 9000, IPv6, transit)
43
44 return parseErrorFiles(v4, v6, v4j, v6j)
45}
46
47func fetchParseRoutes(rsNum, asn int, ipv IPVersion, forJumbo bool) (*RouteFile, error) {
48 mtu := 1500
49 if forJumbo {
50 mtu = 9000
51 }
52
53 f, err := fetchFile(rsPrefixUrl, rsNum, mtu, ipv, asn, mtu, ipv)
54 if err != nil {
55 return nil, err
56 }
57
58 return parseRoutesFile(f), nil
59}
60
61func fetchParseASSet(asn int, ipv IPVersion) (*ASSet, error) {
62 f, err := fetchFile(asnUrl, asn, ipv)
63 if err != nil {
64 return nil, err
65 }
66 return parseASSetFile(f), nil
67}
68
69func fetchParsePrefixList(asn int, ipv IPVersion, forAsSet bool) (*PrefixSet, error) {
70 url := pfxUrl
71 if forAsSet {
72 url = asSetUrl
73 }
74
75 f, err := fetchFile(url, asn, ipv)
76 if err != nil {
77 return nil, err
78 }
79 return parsePrefixSetFile(f), nil
80}
81
82func fetchParseROAFile(asn int) (*ROASet, error) {
83 f, err := fetchFile(roaUrl, asn)
84 if err != nil {
85 return nil, err
86 }
87 return parseROAFile(f), nil
88}
89
90func fetchFile(urlTpl string, args ...interface{}) ([]string, error) {
91 r, err := http.Get(fmt.Sprintf(urlTpl, args...))
92 if err != nil {
93 return nil, err
94 }
95 defer r.Body.Close()
96
97 b, err := ioutil.ReadAll(r.Body)
98 if err != nil {
99 return nil, err
100 }
101
102 return strings.Split(string(b), "\n"), nil
103}
104
105func parseROAFile(lines []string) *ROASet {
106 rs := &ROASet{
107 ROAS: []ROA{},
108 }
109
110 for _, line := range lines {
111 if strings.HasPrefix(line, "Timestamp:") {
112 rs.Timestamp = *mustParseLongTime(line[11:])
113 continue
114 } else if strings.HasPrefix(line, "Trust Anchor") {
115 continue
116 } else if strings.TrimSpace(line) == "" {
117 continue
118 } else {
119 lp := strings.Split(line, "|")
120 rs.ROAS = append(rs.ROAS, ROA{
121 TrustAnchor: strings.TrimSpace(lp[0]),
122 ASN: mustParseInt(lp[1]),
123 MaxLength: mustParseInt(lp[2]),
124 Prefix: *parseIPNetFromCIDR(lp[3]),
125 })
126 }
127 }
128
129 return rs
130}
131
132func parsePrefixSetFile(lines []string) *PrefixSet {
133 ps := &PrefixSet{
134 Prefixes: []net.IPNet{},
135 }
136
137 for _, line := range lines {
138 if strings.HasPrefix(line, "Timestamp:") {
139 ps.Timestamp = *mustParseLongTime(line[11:])
140 continue
141 } else if strings.HasPrefix(line, "as-set:") {
142 ps.ASSet = &strings.Split(line, " ")[1]
143 continue
144 } else if strings.TrimSpace(line) == "" {
145 continue
146 } else {
147 // Data formatting error on SIX side
148 if strings.HasSuffix(line, ",") {
149 line = line[:len(line)-1]
150 }
151 ps.Prefixes = append(ps.Prefixes, *parseIPNetFromCIDR(line))
152 }
153 }
154
155 return ps
156}
157
158func parseASSetFile(lines []string) *ASSet {
159 as := &ASSet{
160 ASNumbers: []int{},
161 }
162
163 for _, line := range lines {
164 if strings.HasPrefix(line, "Timestamp:") {
165 as.Timestamp = *mustParseLongTime(line[11:])
166 continue
167 } else if strings.HasPrefix(line, "as-set:") {
168 as.Name = strings.Split(line, " ")[1]
169 continue
170 } else if strings.TrimSpace(line) == "" {
171 continue
172 } else {
173 as.ASNumbers = append(as.ASNumbers, mustParseInt(line))
174 }
175 }
176
177 return as
178}
179
180func parseErrorFiles(v4, v6, v4j, v6j []string) *Errors {
181 e := &Errors{}
182
183 if v4 != nil {
184 parseErrorFile(v4, e, IPv4, false)
185 }
186 if v6 != nil {
187 parseErrorFile(v6, e, IPv6, false)
188 }
189 if v4j != nil {
190 parseErrorFile(v4j, e, IPv4, true)
191 }
192 if v6j != nil {
193 parseErrorFile(v6j, e, IPv6, true)
194 }
195
196 return e
197}
198
199func getTarget(e *Errors, ipv IPVersion, jumbo bool) *ErrorRecords {
200 if ipv == IPv4 && jumbo {
201 if e.IPv4Jumbo == nil {
202 e.IPv4Jumbo = NewErrorRecords()
203 }
204 return e.IPv4Jumbo
205 } else if ipv == IPv6 && jumbo {
206 if e.IPv6Jumbo == nil {
207 e.IPv6Jumbo = NewErrorRecords()
208 }
209 return e.IPv6Jumbo
210 } else if ipv == IPv4 {
211 if e.IPv4 == nil {
212 e.IPv4 = NewErrorRecords()
213 }
214 return e.IPv4
215 } else if ipv == IPv6 {
216 if e.IPv6 == nil {
217 e.IPv6 = NewErrorRecords()
218 }
219 return e.IPv6
220 } else {
221 // This should not be possible
222 panic("Could not find target")
223 }
224}
225
226func parseErrorFile(lines []string, e *Errors, ipv IPVersion, jumbo bool) {
227 t := getTarget(e, ipv, jumbo)
228
229 for _, line := range lines {
230 l := parseErrorLine(line)
231 if l == nil {
232 // Skip blank lines, which sometimes happen
233 if strings.TrimSpace(line) == "" {
234 continue
235 } else {
236 t.UnparsableLines = append(t.UnparsableLines, line)
237 }
238 } else {
239 t.Records = append(t.Records, *l)
240 }
241 }
242}
243
244func parseErrorLine(line string) *ErrorRecord {
245 res := errorLineRegex.FindAllStringSubmatch(line, -1)
246 if len(res) == 0 {
247 return nil
248 } else {
249 row := res[0][1:]
250 return &ErrorRecord{
251 Timestamp: *mustParseLongTimeNoZone(row[0]),
252 ASN: mustParseInt(row[1]),
253 MTU: mustParseInt(row[2]),
254 Version: IPVersion(mustParseInt(row[3])),
255 NetworkName: row[4],
256 Network: *parseIPNetFromCIDR(row[5]),
257 Router: *parseIP(row[6]),
258 Path: parseASPath(row[7]),
259 Message: row[8],
260 }
261 }
262}
263
264func parseRoutesFile(lines []string) *RouteFile {
265 ps := &RouteFile{
266 Lines: []string{},
267 }
268 for _, line := range lines {
269 if strings.HasPrefix(line, "Timestamp:") {
270 ps.Timestamp = *mustParseLongTime(line[11:])
271 continue
272 } else if strings.TrimSpace(line) == "" {
273 continue
274 } else {
275 ps.Lines = append(ps.Lines, line)
276 }
277 }
278 return ps
279}
diff --git a/six/info_file_parser_test.go b/six/info_file_parser_test.go
new file mode 100644
index 0000000..9994e6f
--- /dev/null
+++ b/six/info_file_parser_test.go
@@ -0,0 +1,256 @@
1// SPDX-License-Identifier: GPL-2.0-only
2// Copyright (C) 2020 Michael Crute <mike@crute.us>. All rights reserved.
3//
4// Use of this source code is governed by a license that can be found in the
5// LICENSE file.
6
7package six
8
9import (
10 "net"
11 "strings"
12 "testing"
13 "time"
14
15 "github.com/stretchr/testify/assert"
16)
17
18var testDate = time.Date(2020, 02, 12, 19, 43, 23, 0, time.UTC)
19
20const roaTest = `
21Timestamp: 2020-02-12 19:43:23 UTC
22Trust Anchor | ASN | Max Length | Prefix
23 APNIC | 64496 | 24 | 192.0.2.0/24
24 RIPE | 64496 | 24 | 198.51.100.0/24
25 RIPE | 64496 | 48 | 2001:db8:3a80::/48
26 APNIC | 64496 | 48 | 2001:db8:3a81::/48
27`
28
29func parseCidr(c string) net.IPNet {
30 _, n, _ := net.ParseCIDR(c)
31 return *n
32}
33
34func TestROAParsing(t *testing.T) {
35 rs := parseROAFile(strings.Split(roaTest, "\n"))
36
37 matrix := []struct {
38 TrustAnchor string
39 ASN int
40 MaxLength int
41 Prefix net.IPNet
42 }{
43 {"APNIC", 64496, 24, parseCidr("192.0.2.0/24")},
44 {"RIPE", 64496, 24, parseCidr("198.51.100.0/24")},
45 {"RIPE", 64496, 48, parseCidr("2001:db8:3a80::/48")},
46 {"APNIC", 64496, 48, parseCidr("2001:db8:3a81::/48")},
47 }
48
49 for i, tv := range matrix {
50 assert.EqualValues(t, tv, rs.ROAS[i])
51 }
52
53 assert.Equal(t, rs.Timestamp, testDate)
54}
55
56const prefixListv4 = `
57Timestamp: 2020-02-12 19:43:23 UTC
58192.0.2.0/24
59198.51.100.0/24
60`
61
62func TestPrefixSetv4Parsing(t *testing.T) {
63 r := parsePrefixSetFile(strings.Split(prefixListv4, "\n"))
64
65 assert.Equal(t, r.Timestamp, testDate)
66 assert.Equal(t, r.Prefixes[0], parseCidr("192.0.2.0/24"))
67 assert.Equal(t, r.Prefixes[1], parseCidr("198.51.100.0/24"))
68}
69
70const prefixListv6 = `
71Timestamp: 2020-02-12 19:43:23 UTC
722001:db8:3a80::/48
732001:db8:3a81::/48
74`
75
76func TestPrefixSetv6Parsing(t *testing.T) {
77 r := parsePrefixSetFile(strings.Split(prefixListv6, "\n"))
78
79 assert.Equal(t, r.Timestamp, testDate)
80 assert.Equal(t, r.Prefixes[0], parseCidr("2001:db8:3a80::/48"))
81 assert.Equal(t, r.Prefixes[1], parseCidr("2001:db8:3a81::/48"))
82}
83
84const asPrefixListv4 = `
85as-set: AS-TEST
86Timestamp: 2020-02-12 19:43:23 UTC
87192.0.2.0/24
88198.51.100.0/24
89198.51.100.0/24,
90`
91
92func TestPrefixSetASNv4Parsing(t *testing.T) {
93 r := parsePrefixSetFile(strings.Split(asPrefixListv4, "\n"))
94
95 assert.Equal(t, r.Timestamp, testDate)
96 assert.Equal(t, *r.ASSet, "AS-TEST")
97 assert.Equal(t, r.Prefixes[0], parseCidr("192.0.2.0/24"))
98 assert.Equal(t, r.Prefixes[1], parseCidr("198.51.100.0/24"))
99 assert.Equal(t, r.Prefixes[2], parseCidr("198.51.100.0/24"))
100}
101
102const asPrefixListv6 = `
103as-set: AS-TEST
104Timestamp: 2020-02-12 19:43:23 UTC
1052001:db8:3a80::/48
1062001:db8:3a81::/48
1072001:db8:3a81::/48,
108`
109
110func TestPrefixSetASNv6Parsing(t *testing.T) {
111 r := parsePrefixSetFile(strings.Split(asPrefixListv6, "\n"))
112
113 assert.Equal(t, r.Timestamp, testDate)
114 assert.Equal(t, *r.ASSet, "AS-TEST")
115 assert.Equal(t, r.Prefixes[0], parseCidr("2001:db8:3a80::/48"))
116 assert.Equal(t, r.Prefixes[1], parseCidr("2001:db8:3a81::/48"))
117 assert.Equal(t, r.Prefixes[2], parseCidr("2001:db8:3a81::/48"))
118}
119
120const asSetFile = `
121as-set: AS-TEST
122Timestamp: 2020-02-12 19:43:23 UTC
12364496
12464497
12564498
126`
127
128func TestASSetParsing(t *testing.T) {
129 r := parseASSetFile(strings.Split(asSetFile, "\n"))
130
131 assert.Equal(t, r.Timestamp, testDate)
132 assert.Equal(t, r.Name, "AS-TEST")
133 assert.Equal(t, r.ASNumbers[0], 64496)
134 assert.Equal(t, r.ASNumbers[1], 64497)
135 assert.Equal(t, r.ASNumbers[2], 64498)
136}
137
138const errorFilev4 = `
1392020-02-12 19:43:23 <INFO> 64496:1:1500:v4:Testnet: 192.0.2.0/24 from 192.0.2.1 (path 64496 64497 64497 64497 64499): AS64499 not member of IRR as-set object! Dropping.
1402020-02-12 19:43:23 <INFO> 64496:1:1500:v4:Testnet: 198.51.100.0/24 from 192.0.2.1: bgp_path(path 64496 64497 64498 64498 64499) matches pdb_never_via_route_servers_ASNs! Dropping.
1412020-02-12 19:43:23 <INFO> 64496:1:1500:v4:Testnet: 192.0.2.0/26 from 192.0.2.1 (path 64496): RPKI ROA_INVALID! Dropping.
142`
143
144const errorFilev6 = `
1452020-02-12 19:43:23 <INFO> 64496:1:1500:v6:Testnet: 2001:db8::/16 from 2001:db8::1: matches bogon_prefixes! Dropping.
1462020-02-12 19:43:23 <INFO> 64496:1:1500:v6:Testnet: 2001:db8::/24 from 2001:db8::1: bgp_path(path 64496 64497 64497 64497 64497 64497 64497 64497 64497 64498) matches pdb_never_via_route_servers_ASNs! Dropping.
1472020-02-12 19:43:23 <INFO> 64496:1:1500:v6:Testnet: 2001:db8::/32 from 2001:db8::1 (path 64496 64497 64498 64498 64499): AS64499 not member of IRR as-set object! Dropping.
148`
149
150// Last line intentionally not parsable
151const errorFilev4j = `
1522020-02-12 19:43:23 <INFO> 64496:1:9000:v4:Testnet: 192.0.2.0/24 from 192.0.2.1: bgp_path(path 64496 64498 { 75757 }) matches bogon_ASNs! Dropping.
1532020-02-12 19:43:23 <INFO> 64496:1:9000:v4:Testnet: 198.51.100.0/24 from 192.0.2.1: matches bogon_prefixes! Dropping.
1542 20-02-12 19:43:23 <INFO> 64496:1:9000:v4:Testnet: 192.0.2.0/26 from 192.0.2.1: matches bogon_prefixes! Dropping.
155`
156
157// Last line intentionally not parsable
158const errorFilev6j = `
1592020-02-12 19:43:23 <INFO> 64496:1:9000:v6:Testnet: 2001:db8::/32 from 2001:db8::1 (path 64496 64497 64499): no IRR route object found! Transit_dropping.
1602020-02-12 19:43:23 <INFO> 64496:1:9000:v6:Testnet: 2001:db8::/36 from 2001:db8::1 (path 64496 64498 64499): no IRR route object found! Transit_dropping.
1612020-02-12 19:43:23 <INFO> 64496:1:9000:v6:Testnet: 2001:db8::/48 from 2001:db8::1 (path 64496 64498 64498 64498 64499): RPKI ROA_INVALID! Transit_dropping.
1622 20-02-12 19:43:23 <INFO> 64496:1:9000:v6:Testnet: 2001:db8::/48 from 2001:db8::1 (path 64496 64498 64498 64498 64499): RPKI ROA_INVALID! Transit_dropping.
163`
164
165func TestParsingErrorFile(t *testing.T) {
166 e := parseErrorFiles(
167 strings.Split(errorFilev4, "\n"),
168 strings.Split(errorFilev6, "\n"),
169 strings.Split(errorFilev4j, "\n"),
170 strings.Split(errorFilev6j, "\n"))
171
172 v4t := []struct {
173 Timestamp time.Time
174 ASN int
175 MTU int
176 Version IPVersion
177 NetworkName string
178 Network net.IPNet
179 Router net.IP
180 Path []int
181 Message string
182 }{
183 {testDate, 64496, 1500, IPv4, "Testnet", parseCidr("192.0.2.0/24"), net.ParseIP("192.0.2.1"), []int{64496, 64497, 64497, 64497, 64499}, "AS64499 not member of IRR as-set object! Dropping."},
184 {testDate, 64496, 1500, IPv4, "Testnet", parseCidr("198.51.100.0/24"), net.ParseIP("192.0.2.1"), []int{64496, 64497, 64498, 64498, 64499}, "matches pdb_never_via_route_servers_ASNs! Dropping."},
185 {testDate, 64496, 1500, IPv4, "Testnet", parseCidr("192.0.2.0/26"), net.ParseIP("192.0.2.1"), []int{64496}, "RPKI ROA_INVALID! Dropping."},
186 }
187
188 for i, tv := range v4t {
189 assert.EqualValues(t, tv, e.IPv4.Records[i])
190 }
191
192 v6t := []struct {
193 Timestamp time.Time
194 ASN int
195 MTU int
196 Version IPVersion
197 NetworkName string
198 Network net.IPNet
199 Router net.IP
200 Path []int
201 Message string
202 }{
203 {testDate, 64496, 1500, IPv6, "Testnet", parseCidr("2001:db8::/16"), net.ParseIP("2001:db8::1"), []int{}, "matches bogon_prefixes! Dropping."},
204 {testDate, 64496, 1500, IPv6, "Testnet", parseCidr("2001:db8::/24"), net.ParseIP("2001:db8::1"), []int{64496, 64497, 64497, 64497, 64497, 64497, 64497, 64497, 64497, 64498}, "matches pdb_never_via_route_servers_ASNs! Dropping."},
205 {testDate, 64496, 1500, IPv6, "Testnet", parseCidr("2001:db8::/32"), net.ParseIP("2001:db8::1"), []int{64496, 64497, 64498, 64498, 64499}, "AS64499 not member of IRR as-set object! Dropping."},
206 }
207
208 for i, tv := range v6t {
209 assert.EqualValues(t, tv, e.IPv6.Records[i])
210 }
211
212 v4jt := []struct {
213 Timestamp time.Time
214 ASN int
215 MTU int
216 Version IPVersion
217 NetworkName string
218 Network net.IPNet
219 Router net.IP
220 Path []int
221 Message string
222 }{
223 {testDate, 64496, 9000, IPv4, "Testnet", parseCidr("192.0.2.0/24"), net.ParseIP("192.0.2.1"), []int{64496, 64498, 75757}, "matches bogon_ASNs! Dropping."},
224 {testDate, 64496, 9000, IPv4, "Testnet", parseCidr("198.51.100.0/24"), net.ParseIP("192.0.2.1"), []int{}, "matches bogon_prefixes! Dropping."},
225 }
226
227 for i, tv := range v4jt {
228 assert.EqualValues(t, tv, e.IPv4Jumbo.Records[i])
229 }
230
231 badLinev4j := "2 20-02-12 19:43:23 <INFO> 64496:1:9000:v4:Testnet: 192.0.2.0/26 from 192.0.2.1: matches bogon_prefixes! Dropping."
232 assert.Equal(t, badLinev4j, e.IPv4Jumbo.UnparsableLines[0])
233
234 v6jt := []struct {
235 Timestamp time.Time
236 ASN int
237 MTU int
238 Version IPVersion
239 NetworkName string
240 Network net.IPNet
241 Router net.IP
242 Path []int
243 Message string
244 }{
245 {testDate, 64496, 9000, IPv6, "Testnet", parseCidr("2001:db8::/32"), net.ParseIP("2001:db8::1"), []int{64496, 64497, 64499}, "no IRR route object found! Transit_dropping."},
246 {testDate, 64496, 9000, IPv6, "Testnet", parseCidr("2001:db8::/36"), net.ParseIP("2001:db8::1"), []int{64496, 64498, 64499}, "no IRR route object found! Transit_dropping."},
247 {testDate, 64496, 9000, IPv6, "Testnet", parseCidr("2001:db8::/48"), net.ParseIP("2001:db8::1"), []int{64496, 64498, 64498, 64498, 64499}, "RPKI ROA_INVALID! Transit_dropping."},
248 }
249
250 for i, tv := range v6jt {
251 assert.EqualValues(t, tv, e.IPv6Jumbo.Records[i])
252 }
253
254 badLinev6j := "2 20-02-12 19:43:23 <INFO> 64496:1:9000:v6:Testnet: 2001:db8::/48 from 2001:db8::1 (path 64496 64498 64498 64498 64499): RPKI ROA_INVALID! Transit_dropping."
255 assert.Equal(t, badLinev6j, e.IPv6Jumbo.UnparsableLines[0])
256}
diff --git a/six/participant.go b/six/participant.go
new file mode 100644
index 0000000..e8150bb
--- /dev/null
+++ b/six/participant.go
@@ -0,0 +1,224 @@
1// SPDX-License-Identifier: GPL-2.0-only
2// Copyright (C) 2020 Michael Crute <mike@crute.us>. All rights reserved.
3//
4// Use of this source code is governed by a license that can be found in the
5// LICENSE file.
6
7package six
8
9import (
10 "net"
11 "time"
12)
13
14// IP protocol version number
15type IPVersion int
16
17const (
18 IPv6 IPVersion = 6
19 IPv4 = 4
20)
21
22// IPv4 and IPv6 union type, contains a full network for IPv4 and IPv6
23// addresses
24type Addresses struct {
25 IPv4 *net.IPNet
26 IPv6 *net.IPNet
27}
28
29// IRR statistical data. Contains the counts of various IRR data for a
30// participant.
31type IRRData struct {
32 PrefixCount int
33 ASNCount int
34 ASSetCount int
35}
36
37// Set of prefixes, optionally with an AS-SET name that a participant has
38// registered with the exchange.
39type PrefixSet struct {
40 Timestamp time.Time
41 ASSet *string
42 Prefixes []net.IPNet
43}
44
45// Set of AS numbers with an AS-SET that the participant has registered with
46// the exchange.
47type ASSet struct {
48 Timestamp time.Time
49 Name string
50 ASNumbers []int
51}
52
53// Set of ROA record summaries that have been parsed from the raw ROA data for
54// a participant.
55type ROASet struct {
56 Timestamp time.Time
57 ROAS []ROA
58}
59
60// Individual ROA summary
61type ROA struct {
62 TrustAnchor string
63 ASN int
64 MaxLength int
65 Prefix net.IPNet
66}
67
68// A SIX participant is a record of an organization's connection to the Seattle
69// Internet Exchange. It contains information about their connect, the prefixes
70// and ASes that they have registered with the exchange, and their usage of the
71// route server.
72//
73// Not all data exists for all participants, and the data that does exist isn't
74// always in a nice clean format. The parsers attempt to make sense of whatever
75// is available but some data may be missing or inconsistent.
76//
77// The default contains summary data for the participant and more detailed data
78// can be fetched with the methods attached to this struct as well as to the
79// RouteServer structs, if the participant is using the route server.
80type SIXParticipant struct {
81 Organization string
82 URL string
83 ASN int
84 Speed int // Connection speed in Mbit/s
85 Switch string // Connected Switch or Location
86 Contact string // Contact Email
87 Comment string
88 IsConnected bool
89 IsVoter bool
90 Update *time.Time
91 Options []string // Currently: IPv6 and MTU9k
92 PeeringPolicy string
93 ROACount int
94 PeeringDBPrefixCountv4 int
95 PeeringDBPrefixCountv6 int
96 Addresses Addresses
97 JumboAddresses *Addresses
98 IRRv4 IRRData
99 IRRv6 IRRData
100 RouteServer2 *RouteServer
101 RouteServer3 *RouteServer
102}
103
104// Get the set of prefixes that the participant has registered.
105//
106// This does an HTTP fetch to get the detailed file.
107func (p *SIXParticipant) GetPrefixes(ip IPVersion) (*PrefixSet, error) {
108 return fetchParsePrefixList(p.ASN, ip, false)
109}
110
111// Get the prefix to AS associations that the participant has registered.
112//
113// This does an HTTP fetch to get the detailed file.
114func (p *SIXParticipant) GetASPrefixes(ip IPVersion) (*PrefixSet, error) {
115 return fetchParsePrefixList(p.ASN, ip, true)
116}
117
118// Get the AS sets that the participant has registered
119//
120// This does an HTTP fetch to get the detailed file.
121func (p *SIXParticipant) GetASSet(ip IPVersion) (*ASSet, error) {
122 return fetchParseASSet(p.ASN, ip)
123}
124
125// Get the ROA set and more detailed records for the ROAs that the participant
126// has registered. This does not fetch the raw ROAs, just the summary that the
127// exchange has computed.
128//
129// This does an HTTP fetch to get the detailed file.
130func (p *SIXParticipant) GetROASet() (*ROASet, error) {
131 return fetchParseROAFile(p.ASN)
132}
133
134// List of lines from the file containing route data dumped from the route
135// server
136type RouteFile struct {
137 Timestamp time.Time
138 Lines []string
139}
140
141// Lists of errors that the route server has exported. These are parsed for
142// more details if possible, but if that fails the attached ErrorRecords object
143// contains a list of un-parsable lines for consumption.
144type Errors struct {
145 IPv4 *ErrorRecords
146 IPv6 *ErrorRecords
147 IPv4Jumbo *ErrorRecords
148 IPv6Jumbo *ErrorRecords
149}
150
151// List of error records for route server errors. Also contains a list of lines
152// that could not be parsed by the parser.
153type ErrorRecords struct {
154 Records []ErrorRecord
155 UnparsableLines []string
156}
157
158// Create a new ErrorRecords struct
159func NewErrorRecords() *ErrorRecords {
160 return &ErrorRecords{
161 Records: []ErrorRecord{},
162 UnparsableLines: []string{},
163 }
164}
165
166// Contains the parsed data for a route server error.
167type ErrorRecord struct {
168 Timestamp time.Time
169 ASN int
170 MTU int
171 Version IPVersion
172 NetworkName string
173 Network net.IPNet
174 Router net.IP
175 Path []int
176 Message string
177}
178
179// Data about the participant's connection to a route server. If the
180// participant is making use of the route servers this will always contain some
181// basic statistics about their connection. More detailed information can be
182// obtained by calling the methods attached to this struct.
183type RouteServer struct {
184 Number int
185 IPv4 RouteServerStats
186 IPv6 RouteServerStats
187 IPv4Jumbo RouteServerStats
188 IPv6Jumbo RouteServerStats
189 asn int
190}
191
192// Route server statistics. Contains the counts of various types of route
193// server entries.
194type RouteServerStats struct {
195 Prefixes int
196 Errors int
197 TransitErrors int
198}
199
200// Gets the list of routes being advertised by the participant. Note that this
201// file is the raw lines from the route server and no attempt has been made to
202// parse these lines. The lines are in BIRD format.
203//
204// This does an HTTP fetch to get the detailed file.
205func (s *RouteServer) GetRoutes(ip IPVersion, jumbo bool) (*RouteFile, error) {
206 return fetchParseRoutes(s.Number, s.asn, ip, jumbo)
207}
208
209// Gets the errors returned by the route server for all IP protocols and all
210// VLANs to which the participant is connected. If the participant is using
211// multiple route servers, this data is scoped to the current route server.
212//
213// This does an HTTP fetch to get the detailed file.
214func (s *RouteServer) GetErrors() (*Errors, error) {
215 return fetchParseRSErrors(s.Number, s.asn, false), nil
216}
217
218// Get a list of transit errors from the route server. Otherwise return value
219// and behavior is identical to GetErrors.
220//
221// This does an HTTP fetch to get the detailed file.
222func (s *RouteServer) GetTransitErrors() (*Errors, error) {
223 return fetchParseRSErrors(s.Number, s.asn, true), nil
224}
diff --git a/six/participant_builder.go b/six/participant_builder.go
new file mode 100644
index 0000000..f18a6d1
--- /dev/null
+++ b/six/participant_builder.go
@@ -0,0 +1,113 @@
1// SPDX-License-Identifier: GPL-2.0-only
2// Copyright (C) 2020 Michael Crute <mike@crute.us>. All rights reserved.
3//
4// Use of this source code is governed by a license that can be found in the
5// LICENSE file.
6
7package six
8
9import (
10 "fmt"
11 "strings"
12)
13
14// Create a new SIXParticipant struct from a map of data that was parsed from
15// the participant CSV file. This assumes the column headers from that CSV file
16// so it will not work with the other data formats avaiable.
17//
18// This uses the CSV file because it's both the most rich source of data and
19// the easiest to parse.
20func NewSIXParticipantFromData(d map[string]string) *SIXParticipant {
21 r := &SIXParticipant{
22 Organization: d["Organization"],
23 URL: d["URL"],
24 ASN: mustParseInt(d["ASN"]),
25 Speed: mustParseInt(d["Speed"]),
26 Switch: d["Switch"],
27 Contact: d["Contact"],
28 Comment: d["Comment"],
29 IsConnected: parseYesNo(d["Conn?"]),
30 IsVoter: parseYesNo(d["Voter?"]),
31 Update: mustParseTime(d["Update"]),
32 Options: strings.Split(d["Options"], " "),
33 PeeringPolicy: d["Policy"],
34 ROACount: mustParseInt(d["rpki:roa"]),
35 PeeringDBPrefixCountv4: mustParseInt(d["pdb:v4"]),
36 PeeringDBPrefixCountv6: mustParseInt(d["pdb:v6"]),
37 Addresses: Addresses{
38 IPv4: parseIPNetFromCIDR(d["IPv4"]),
39 IPv6: parseIPNetFromCIDR(d["IPv6"]),
40 },
41 IRRv4: IRRData{
42 PrefixCount: mustParseInt(d["irr:p4"]),
43 ASNCount: mustParseInt(d["irr:a4"]),
44 ASSetCount: mustParseInt(d["irr:ap4"]),
45 },
46 IRRv6: IRRData{
47 PrefixCount: mustParseInt(d["irr:p6"]),
48 ASNCount: mustParseInt(d["irr:a6"]),
49 ASSetCount: mustParseInt(d["irr:ap6"]),
50 },
51 RouteServer2: getRSData(2, d),
52 RouteServer3: getRSData(3, d),
53 }
54
55 // Not all participants use the MTU9k VLAN
56 ja4 := parseIPNetFromCIDR(d["Jumbo IPv4"])
57 ja6 := parseIPNetFromCIDR(d["Jumbo IPv6"])
58 if ja4 != nil && ja6 != nil {
59 r.JumboAddresses = &Addresses{IPv4: ja4, IPv6: ja6}
60 }
61
62 return r
63}
64
65func getRSData(server int, d map[string]string) *RouteServer {
66 // Extract all the data and determine if it's all empty strings, if so then
67 // the participant isn't using the route server. If any data is not empty
68 // then they are. Do integer conversion afterward to avoid ambiguity about
69 // zero vs empty string.
70 pd := []string{
71 d[fmt.Sprintf("rs%d:v4", server)],
72 d[fmt.Sprintf("err%d:v4", server)],
73 d[fmt.Sprintf("xerr%d:v4", server)],
74 d[fmt.Sprintf("rs%d:v6", server)],
75 d[fmt.Sprintf("err%d:v6", server)],
76 d[fmt.Sprintf("xerr%d:v6", server)],
77 d[fmt.Sprintf("rs%d:v4j", server)],
78 d[fmt.Sprintf("err%d:v4j", server)],
79 d[fmt.Sprintf("xerr%d:v4j", server)],
80 d[fmt.Sprintf("rs%d:v6j", server)],
81 d[fmt.Sprintf("err%d:v6j", server)],
82 d[fmt.Sprintf("xerr%d:v6j", server)],
83 }
84
85 if allEmpty(pd) {
86 return nil
87 }
88
89 return &RouteServer{
90 Number: server,
91 asn: mustParseInt(d["ASN"]),
92 IPv4: RouteServerStats{
93 Prefixes: mustParseInt(pd[0]),
94 Errors: mustParseInt(pd[1]),
95 TransitErrors: mustParseInt(pd[2]),
96 },
97 IPv6: RouteServerStats{
98 Prefixes: mustParseInt(pd[3]),
99 Errors: mustParseInt(pd[4]),
100 TransitErrors: mustParseInt(pd[5]),
101 },
102 IPv4Jumbo: RouteServerStats{
103 Prefixes: mustParseInt(pd[6]),
104 Errors: mustParseInt(pd[7]),
105 TransitErrors: mustParseInt(pd[8]),
106 },
107 IPv6Jumbo: RouteServerStats{
108 Prefixes: mustParseInt(pd[9]),
109 Errors: mustParseInt(pd[10]),
110 TransitErrors: mustParseInt(pd[11]),
111 },
112 }
113}
© 2015 Mike Crute