Ask HN: How would you bootstrap a telco?
47 comments
Having deployed extensive optical networks both aerially and underground throughout Iowa, my recommendation would be to locate a community that would value your contribution as a starting point. I would then deploy physical plant for specific customers overbuilding the fiber optics (e.g. if the customer wants 12 fibers, deploy 48 selling 12 and keeping 36 for your own future use).
In my experience VERY few people in the optical networking space understand how to engineer a municipal fiber optic network - they invest thousands of dollars per customer when it can be done with best practices for well under $1k per customer (let me just say that 802.1w RSTP is your friend). You need to combine all the different ISO networking layers into a SINGLE business model (ie. physical plant and Ethernet/VLAN circuits all by the SAME engineer not by different departments, otherwise things get unnecessarily over-engineered).
Even more important than the technical engineering is the financial engineering. Once you understand you will never produce more capital from selling customers than it will cost to provision those customers, you need to consider more advanced financial engineering models - the one that worked great for the cellular industry and several optical networks is commercial paper. Find a lawyer that REALLY understands commercial paper - then every contract signed by a customer actually IS cash and does not need to be converted to cash. It's one of the only ways I know of releasing capital invested into physical plant.
Good luck!
In my experience VERY few people in the optical networking space understand how to engineer a municipal fiber optic network - they invest thousands of dollars per customer when it can be done with best practices for well under $1k per customer (let me just say that 802.1w RSTP is your friend). You need to combine all the different ISO networking layers into a SINGLE business model (ie. physical plant and Ethernet/VLAN circuits all by the SAME engineer not by different departments, otherwise things get unnecessarily over-engineered).
Even more important than the technical engineering is the financial engineering. Once you understand you will never produce more capital from selling customers than it will cost to provision those customers, you need to consider more advanced financial engineering models - the one that worked great for the cellular industry and several optical networks is commercial paper. Find a lawyer that REALLY understands commercial paper - then every contract signed by a customer actually IS cash and does not need to be converted to cash. It's one of the only ways I know of releasing capital invested into physical plant.
Good luck!
On top of that: avoid engineering consulting firms. Hire an engineer that can stamp and sign off on plans to do the work for you. Pole surveying is not all that expensive to do yourself, and there are multiple technology solutions in that space now that didn't exist 10+ years ago. The first engineering consultants I had to pay were billing $6/m. Doing it myself with a P.Eng reviewing is under $1/m.
I cannot agree enough. Design engineering of an FTTH solution is not the domain of a single engineering discipline. If you "locate a community that would value your contribution as a starting point", that city council will move heaven and hell to ensure you have the rights to enhance their town. If you do everything underground, then the city can usually sign off on your easement rights and access (plus they will give you the friendly coordination of the water and sewer department and maybe other local utilities). Going aerially will be quicker and cheaper, but it will be fraught with more potential obstacles from the incumbents. Directional boring, plowing and trenching have become commodity priced construction costs ($ per ft), with massive competition. Many underground construction firms will give you the credibility you need to work responsibly and with accountability (bonding and liability insurance).
Once the physical plant is design engineered, your Ethernet and Internet engineering is virtually unregulated. You will want specialists to set up your BGP constellation and other things that will only be done once, but if you have a good working knowledge of routing and Q-tagged VLANs and other core Ethernet technologies, you will be amazed how much you can do yourself.
Once the physical plant is design engineered, your Ethernet and Internet engineering is virtually unregulated. You will want specialists to set up your BGP constellation and other things that will only be done once, but if you have a good working knowledge of routing and Q-tagged VLANs and other core Ethernet technologies, you will be amazed how much you can do yourself.
On my in-laws side there were a couple of cousins who each worked for a phone coop in Kasuth County.
They were busily installing fiber to hogging operations back around 2000.
As a byproduct, small town residents were paying something like $15 a month for cable and phone lines with every possible feature.
It totally blew my mind.
They were busily installing fiber to hogging operations back around 2000.
As a byproduct, small town residents were paying something like $15 a month for cable and phone lines with every possible feature.
It totally blew my mind.
This is excellent advice.
(Though I am curious why RSTP is so deserving of a call out - I mean, yes, but there's a lot of basics here that someone starting out would need to know, including things like avoiding fate sharing and basic planning, equipment realities, etc. if laying fiber.)
(Though I am curious why RSTP is so deserving of a call out - I mean, yes, but there's a lot of basics here that someone starting out would need to know, including things like avoiding fate sharing and basic planning, equipment realities, etc. if laying fiber.)
> curious why RSTP is so deserving of a call out
I consider RSTP a core foundational technology for FTTH, not some condiment sprinkled on the network afterwards. Plus it is the Ethernet technology I find most typically missing from FTTH network design.
Let me explain by way of an example: imagine a typical small town, population less than 50,000, with 2.60 people on average per household => less than 19,231 households. Traditional FTTH networks were modeled much like a POTS network in a star pattern with a dedicated optical fiber to each household from a central office. That then evolved (or should I say got bastardized) to a HFC (hybrid fiber coax) model by the MSO's as they already had a coax local loop going to each house. Clearly any kind of centralized distributed FTTH is doomed to financial ruin.
As an alternative, image this: same community. You get the local school district as your first customer (or other geographically dispersed participant), where you deploy a ring of fiber circumnavigating the community, interconnecting every school. As above you over build their fiber, keeping some for yourself. Of course, you will have a peering point (or two) to interconnect 10 gigabit Ethernet with a tier one provider. You now have a ring of optical fibers less than a few miles from every home in town (typically less than a mile). Built on top of RSTP, the ring of fiber gives every node on the network route-diversified redundancy and protection. But not every residential customer is going to sign up with you, so the last thing you want to do is build out to every home. Instead you build out from your ring to those households that do sign up, where every home has a route-diversified optical connection extending the RSTP ring to the customer premise. This topology negates the need for a massive star network emanating from a central office. Literally with a handful of fibers you can provide for an entire community at a standard of engineering and quality of service the MSOs cannot even fathom. Massive reduction in physical plant requirement, massive reduction in capital per customer.
I consider RSTP a core foundational technology for FTTH, not some condiment sprinkled on the network afterwards. Plus it is the Ethernet technology I find most typically missing from FTTH network design.
Let me explain by way of an example: imagine a typical small town, population less than 50,000, with 2.60 people on average per household => less than 19,231 households. Traditional FTTH networks were modeled much like a POTS network in a star pattern with a dedicated optical fiber to each household from a central office. That then evolved (or should I say got bastardized) to a HFC (hybrid fiber coax) model by the MSO's as they already had a coax local loop going to each house. Clearly any kind of centralized distributed FTTH is doomed to financial ruin.
As an alternative, image this: same community. You get the local school district as your first customer (or other geographically dispersed participant), where you deploy a ring of fiber circumnavigating the community, interconnecting every school. As above you over build their fiber, keeping some for yourself. Of course, you will have a peering point (or two) to interconnect 10 gigabit Ethernet with a tier one provider. You now have a ring of optical fibers less than a few miles from every home in town (typically less than a mile). Built on top of RSTP, the ring of fiber gives every node on the network route-diversified redundancy and protection. But not every residential customer is going to sign up with you, so the last thing you want to do is build out to every home. Instead you build out from your ring to those households that do sign up, where every home has a route-diversified optical connection extending the RSTP ring to the customer premise. This topology negates the need for a massive star network emanating from a central office. Literally with a handful of fibers you can provide for an entire community at a standard of engineering and quality of service the MSOs cannot even fathom. Massive reduction in physical plant requirement, massive reduction in capital per customer.
Can you dumb this down for someone who understands Internet networking (incl. internet backbone routing), and vaguely understands shared-medium networking like wi-fi and cellular networks, but who knows next to nothing about coax, and especially nothing about fiber?
What does a fiber loop (bus?) let you do hardware-wise, that a star topology wouldn't? Either way, you're running a piece of fiber to the home from somewhere, and connecting it to your network somehow; and I would assume that it's that connection that's inherently expensive, because of the equipment involved. In star topologies, that's a switch, or a hierarchy of switches. For a shared medium, that's a hub—or, I guess, some kind of direct splice, if that's somehow cheaper in TCO.
Is the goal of using a loop, to avoid the cost of having neighbourhood-distribution-box optical network switches, instead directly splicing customers onto the ring, making the customer's fiber "part of" the shared medium of the loop, and then allowing customer equipment to directly push packets / transmission cells onto the loop?
If so, why does it sound like you're implying that in a traditional FTTH star topology, there wouldn't be per-neighbourhood optical network switches, and instead only one wide-branching layer where all customers' lines terminate directly at one central location? That seems kind of ridiculous. Are optical switches really that expensive, that those are the only two options? Especially when neighbourhood-distribution-box optical switches would be relatively-low fan-in + relatively-low bandwidth, and therefore could be built to use simple low-frequency TDM circuit-switching protocols like coax's ATM?
Also, is client-accessible fiber head-end equipment really trustworthy to do TDM (FDM?) onto a shared fiber medium, without ruining it for everyone else? We don't trust client equipment to do that on enterprise Ethernet networks (if we can at-all help it); we use switches instead of hubs, and we even put VLANs in place to further isolate flows in the merged path for QoS purposes. Is the difference just a matter of a fiber provider only certifying specific (presumably robust) equipment; and of that equipment not being coupled to power delivery issues of an arbitrary host device? Is that enough to get you peace-of-mind that your shared-medium network won't be being blasted with noise from new failing head-end equipment every few days?
What does a fiber loop (bus?) let you do hardware-wise, that a star topology wouldn't? Either way, you're running a piece of fiber to the home from somewhere, and connecting it to your network somehow; and I would assume that it's that connection that's inherently expensive, because of the equipment involved. In star topologies, that's a switch, or a hierarchy of switches. For a shared medium, that's a hub—or, I guess, some kind of direct splice, if that's somehow cheaper in TCO.
Is the goal of using a loop, to avoid the cost of having neighbourhood-distribution-box optical network switches, instead directly splicing customers onto the ring, making the customer's fiber "part of" the shared medium of the loop, and then allowing customer equipment to directly push packets / transmission cells onto the loop?
If so, why does it sound like you're implying that in a traditional FTTH star topology, there wouldn't be per-neighbourhood optical network switches, and instead only one wide-branching layer where all customers' lines terminate directly at one central location? That seems kind of ridiculous. Are optical switches really that expensive, that those are the only two options? Especially when neighbourhood-distribution-box optical switches would be relatively-low fan-in + relatively-low bandwidth, and therefore could be built to use simple low-frequency TDM circuit-switching protocols like coax's ATM?
Also, is client-accessible fiber head-end equipment really trustworthy to do TDM (FDM?) onto a shared fiber medium, without ruining it for everyone else? We don't trust client equipment to do that on enterprise Ethernet networks (if we can at-all help it); we use switches instead of hubs, and we even put VLANs in place to further isolate flows in the merged path for QoS purposes. Is the difference just a matter of a fiber provider only certifying specific (presumably robust) equipment; and of that equipment not being coupled to power delivery issues of an arbitrary host device? Is that enough to get you peace-of-mind that your shared-medium network won't be being blasted with noise from new failing head-end equipment every few days?
The misunderstanding is due to the grandparent getting things wrong.
It’s the building out of fiber to each premise that is the expensive part, not the topology or the equipment. Those are rounding errors. What is expensive is undergrounding the fiber or hanging it on poles.
It’s the building out of fiber to each premise that is the expensive part, not the topology or the equipment. Those are rounding errors. What is expensive is undergrounding the fiber or hanging it on poles.
EMPHASIS > It’s the building out of fiber to each premise that is the expensive part, not the topology or the equipment.
That's it in one sentence. So you want a topology and engineering model that will allow you to reduce your total network cost.
That's it in one sentence. So you want a topology and engineering model that will allow you to reduce your total network cost.
> What does a fiber loop (bus?) let you do hardware-wise, that a star topology wouldn't?
Each site has two links, left and right, that have engineered different paths.
If your link is to home base, chances are your second link to home base is going to take the same path.
Each site has two links, left and right, that have engineered different paths.
If your link is to home base, chances are your second link to home base is going to take the same path.
Spanning tree is an exceptionally bad protocol choice for a ring network. It’s not like you want to create a giant layer 2 network anyway.
I suggest you look into routed networks and redundant GPON topologies.
I suggest you look into routed networks and redundant GPON topologies.
This. I'm not sure why L2 is even being discussed at such a high level. I've worked for Tier 2 carriers and have deployed fiber across multiple states (transport) and have experience dropping out into FTTH solutions. The last thing you want is an unscalable L2 dependent solution that's a royal PITA to debug when things go wrong (and, they will).
IMO one of the core components to think about when doing a buildout is your total solution. I've worked with Infinera in the past for both long haul and PON solutions and the important thing is getting the most return on investment out of your deployed glass. DWDM / band splitting is where you really need to engineer up front.
IMO one of the core components to think about when doing a buildout is your total solution. I've worked with Infinera in the past for both long haul and PON solutions and the important thing is getting the most return on investment out of your deployed glass. DWDM / band splitting is where you really need to engineer up front.
Even if the cited $1k per customer varies by density etc., and perhaps you mix in some fixed-wireless, then how about a business model in which you sell your network to providers (internet providers, telco providers, tv providers...and likely specialty content, business app and service providers...who then sell to the end users)?
You sell B2B (you would need to solve for multi-tenancy), and help facilitate a lot of consumer choice and competition by decoupling the pipe from the services...
You sell B2B (you would need to solve for multi-tenancy), and help facilitate a lot of consumer choice and competition by decoupling the pipe from the services...
Basically, you start by getting giant piles of cash. You then start trying to build out new network assets. Then you run into the incumbents cheating on the permitting processes and utilizing market power to take or retain customers. I've been through this. And the municipalities don't even own up to their obligations under CRTC frameworks and delay MAAs for a year "just because". It's painful.
Becoming the third player is an uphill battle in all regards. Fibre customers are very sticky, and getting people to change is nearly impossible. Bell is particularly awful as a competitor as they steeply discount the first 1-2 years: want gigabit fibre with all the TV channels and phone service for $60/month along with a $200 prepaid Visa gift card? Sure! Then month 13 comes along and the bill becomes $300 per month. As a competitor, there is absolutely no way I can offer a similar package or retain a customer when they take that package. Want that deal as an existing customer? No way!
On top of all that, Canada is a relatively small market that is very spread out geographically. If you want to provide coverage similar to existing carriers, you have to build a lot of infrastructure. Nobody can afford to do that. Plus there are buckets of cash being rained down on other market players under a number of programs, so your competitor may be running on assets that cost 50-100% below your cost to build a similar infrastructure.
The easiest thing for you to do is to get 2 internet connections from different carriers that do not share networks. In most markets the telco and cableco run different networks. My Telus and Bell services continued working fine yesterday. Heck, my 3 different Rogers wavelength services we not impacted at all (although L1 services have other failure modes that take longer to recover from since there is no automatic rerouting as with L2 networks).
I'm pretty sure you really don't want to start down this path...
Becoming the third player is an uphill battle in all regards. Fibre customers are very sticky, and getting people to change is nearly impossible. Bell is particularly awful as a competitor as they steeply discount the first 1-2 years: want gigabit fibre with all the TV channels and phone service for $60/month along with a $200 prepaid Visa gift card? Sure! Then month 13 comes along and the bill becomes $300 per month. As a competitor, there is absolutely no way I can offer a similar package or retain a customer when they take that package. Want that deal as an existing customer? No way!
On top of all that, Canada is a relatively small market that is very spread out geographically. If you want to provide coverage similar to existing carriers, you have to build a lot of infrastructure. Nobody can afford to do that. Plus there are buckets of cash being rained down on other market players under a number of programs, so your competitor may be running on assets that cost 50-100% below your cost to build a similar infrastructure.
The easiest thing for you to do is to get 2 internet connections from different carriers that do not share networks. In most markets the telco and cableco run different networks. My Telus and Bell services continued working fine yesterday. Heck, my 3 different Rogers wavelength services we not impacted at all (although L1 services have other failure modes that take longer to recover from since there is no automatic rerouting as with L2 networks).
I'm pretty sure you really don't want to start down this path...
Heck, my 3 different Rogers wavelength services we not impacted at all (although L1 services have other failure modes that take longer to recover from since there is no automatic rerouting as with L2 networks).
Sort of, our wave circuits will automatically reroute but we had to shell out the $$ for protected waves and then explicitly mention they had to be configured to be auto-revertive as that was not the default behavior for that telecom provider. Then L2 and L3 don't notice when it shifts between the working and protect paths, it happens fast enough the only visible change then is the latency difference between the two.
Sort of, our wave circuits will automatically reroute but we had to shell out the $$ for protected waves and then explicitly mention they had to be configured to be auto-revertive as that was not the default behavior for that telecom provider. Then L2 and L3 don't notice when it shifts between the working and protect paths, it happens fast enough the only visible change then is the latency difference between the two.
In some regards it's better to get another wavelength circuit going to another city to pick up geographically diverse transit.
True, our use case is a bit different as the waves go between PoPs/DCs/Offices so the only diversity can be in the path itself. Faster to have the path switch over and have L3 re-converge only when the working and protected path are unavailable.
As someone in the industry thank you for saying this! The regulatory demands in Canada are much different than elsewhere in the world, and greatly outweigh the technical demands
> The easiest thing for you to do is to get 2 internet connections from different carriers that do not share networks.
This is not as easy as it may seem, as it's common for the carriers to lease from each other in the backbones. Telus and Bell for cellular services share a radio network, each cell site connects to both the Bell and Telus core networks. The big Bell backbone outage in the Toronto area, 6 or 7 years ago, had impacts on all of Bell, Telus, and Rogers networks. As for cellular services we had glitches nationwide that had to be fixed during that backbone outage.
This is not as easy as it may seem, as it's common for the carriers to lease from each other in the backbones. Telus and Bell for cellular services share a radio network, each cell site connects to both the Bell and Telus core networks. The big Bell backbone outage in the Toronto area, 6 or 7 years ago, had impacts on all of Bell, Telus, and Rogers networks. As for cellular services we had glitches nationwide that had to be fixed during that backbone outage.
And even if they don't share networks they might share physical space.
I had an incident where two different trans-Atlantic links from two different providers, shared a 20 mile stretch of conduit.
So, of course, that's exactly where the fiber cut happened.
I had an incident where two different trans-Atlantic links from two different providers, shared a 20 mile stretch of conduit.
So, of course, that's exactly where the fiber cut happened.
In the case of the Rogers outage, other carriers / ISPs were not offline. The Rogers problem appeared to be entirely in L2/L3 -- switches and routers were not configured correctly. The other telcos were not impacted as they run their own routing infrastructure. Basically, make sure your 2 ISPs have their own ASNs and transit providers.
Sorry, I think you missed the point I was trying to share. The large providers all lease infrastructure from each other, so it's unsafe to assume that using two providers creates a total diversity in access. But you're correct, in that failures can happen at different layers, that may isolate outages to one provider or even a subset of customers.
The outage I cited, only occurred in the Bell Wireline Core network, but caused service impacts in Bell, Telus, Rogers and other networks for Cellular, Internet, etc services.
The outage I cited, only occurred in the Bell Wireline Core network, but caused service impacts in Bell, Telus, Rogers and other networks for Cellular, Internet, etc services.
Again, it depends on what you're leasing from other carriers. Yes, if you're purchasing an L2 path from a carrier you're relying on their switches and routers to pass traffic correctly, but you bypass their routers and switches by purchasing an L1 wavelength circuit between 2 points. That's one of the main reasons that wavelength circuits are used (in addition to the guaranteed capacity).
Third party carriers using fibre local loops from another carrier would not have gone down like TPIA users did on Rogers' network yesterday. I strongly believe that access to fibre local loops is needed in Canada. Leased copper local loops enabled plenty of competition between small local ISPs in the 1990s and early 2000s while they could still deliver speeds that were sufficient for internet users of the time. Having a tariff for local fibre circuits similar to the old LDDS tariff would bring back that vibrant competition we once had. Bonus: it's a lot harder for incumbents to put together bogus cost studies as compared to the failed disaggregated regime the CRTC has wasted nearly a decade on now.
Third party carriers using fibre local loops from another carrier would not have gone down like TPIA users did on Rogers' network yesterday. I strongly believe that access to fibre local loops is needed in Canada. Leased copper local loops enabled plenty of competition between small local ISPs in the 1990s and early 2000s while they could still deliver speeds that were sufficient for internet users of the time. Having a tariff for local fibre circuits similar to the old LDDS tariff would bring back that vibrant competition we once had. Bonus: it's a lot harder for incumbents to put together bogus cost studies as compared to the failed disaggregated regime the CRTC has wasted nearly a decade on now.
I think having a good understanding of the ways the Canadian incumbents can screw you out of a business is certainly helpful.
Check out Community Fibre's submission to the CRTC here (the 3.7MB document): https://services.crtc.gc.ca/pub/ListeInterventionList/Docume...
It has lots of specifics about how Bell in particular will use anti-competitive tactics to hamper your business. Also, in many parts of Canada there are simply no other transport providers other than the incumbents that could transport your traffic from your community to an IXP. To stay out of trouble your best bet is usually using a company like Zayo (if you're close to CP rail lines, ex-Allstream fibre) or using incumbents outside of their residential service area (eg. using Shaw for Transport in Ontario if you happen to be close to their fibre).
I really recommend reading the entire 44 pages and then giving Benjamin a call if you have any specific questions. He's a great guy that has experience bootstrapping a fibre ISP (albeit in Ontario, not BC) and can talk about important topics such as pole attachment and dealing with hydro/Bell. I remember our discussion ending up focusing on funding issues with regards to a lack of grant or venture funding of 30+ year horizon infrastructure projects. I think we spoke for almost 2 hours.
It has lots of specifics about how Bell in particular will use anti-competitive tactics to hamper your business. Also, in many parts of Canada there are simply no other transport providers other than the incumbents that could transport your traffic from your community to an IXP. To stay out of trouble your best bet is usually using a company like Zayo (if you're close to CP rail lines, ex-Allstream fibre) or using incumbents outside of their residential service area (eg. using Shaw for Transport in Ontario if you happen to be close to their fibre).
I really recommend reading the entire 44 pages and then giving Benjamin a call if you have any specific questions. He's a great guy that has experience bootstrapping a fibre ISP (albeit in Ontario, not BC) and can talk about important topics such as pole attachment and dealing with hydro/Bell. I remember our discussion ending up focusing on funding issues with regards to a lack of grant or venture funding of 30+ year horizon infrastructure projects. I think we spoke for almost 2 hours.
Thanks for the shout out. =-) I'm glad folks take the time to read some of my lengthy ramblings on telecom.
Would it be fair to say that all these problems would go away if a startup ISP was not trying to lease capacity from their direct (and much larger) competitor?
Or did I misunderstand why permits from Bell are required.
Or did I misunderstand why permits from Bell are required.
CFC doesn't buy transport or transit from Bell as their pricing is ridiculous ($10,800 per month to go 26 km at 10Gbps? No way.). We only rent poles and strand from Bell. As Bell owns a good percentage of the poles in Ontario, it is practically impossible to build aerial plant in rural areas without running into Bell owned support structures. Putting up new poles is a $10k per kilometer added expense, and even more if there are a lot of rocks.
I have worked for a startup ISP almost 10 years ago. I was the first network engineer and I saw the ISP growing from scratch to more than 50k users and then collapse in few months for several terrible management choices.
I don't know how is in Canada, but in Italy you need a lot of regulamentatory works before start to install the first equipment.
Another point to consider if you want to scale up, is to have as much as processes as possible. Without workflows and procedures, the growing complexity will kill your ISP.
> What creative solutions have folks seen?
The idea that the ISP I worked for was simple: you are a very strong local reseller of a big ISP in a very limited area, but you don't have the flexibility in prices and services that you want. So "my ISP" will help you: we install a complete infrastructure in your area, you will pay us a fixed prices every month and we will manage the whole infrastructure on your behalf. As reseller, you have a portal to manage new orders and a support call center to solve the issues. It worked for several years. Feel free to drop me an email if you want more details, (my username) (at) gmail (dot) com
> What creative solutions have folks seen?
The idea that the ISP I worked for was simple: you are a very strong local reseller of a big ISP in a very limited area, but you don't have the flexibility in prices and services that you want. So "my ISP" will help you: we install a complete infrastructure in your area, you will pay us a fixed prices every month and we will manage the whole infrastructure on your behalf. As reseller, you have a portal to manage new orders and a support call center to solve the issues. It worked for several years. Feel free to drop me an email if you want more details, (my username) (at) gmail (dot) com
This is a great read on setting up a WISP (wireless ISP)[1]. It might be US-centric, but has a lot of good resources for the process. Probably smaller-scale than what you're talking about but still a good read.
[1]: https://startyourownisp.com/
[1]: https://startyourownisp.com/
Call me crazy, but I never had issues with Canadian telcos - I start to think I’m some statistical anomaly :). I used Rogers, Bell and Telus for cell - all worked just fine; other than their barely functioning websites I never had a problem. Are they fixing prices? Absolutely, it’s an oligopoly, same as many other things in Canada; I think Rogers has a 40% profit margin, that’s rich if you ask me. Regarding cable internet - there are some cheaper companies such as TekSavvy and CarryTel - they use the big guys’ infra as a backbone, but cost 10-30% cheaper.
My point is that up until this last event I was cognizant of the fact that the prices are too high, but was ok to pay for the quality I got, would have laughed in the face to anyone saying LTE coverage or speed are bad.
As a side note about the outage - I think a lot of people underestimate incompetence in large companies. I often marvel at how for example large Canadian banks are making record profits with many disfuctioning teams in critical positions - a situation that got much worse with the wave of the great resignation. If I were to bet, I’d say some incompetent team broke the infra, so there’s no need to look for conspiracy theories, people are just not competent in their positions and make mistakes when something has to be changed and they have no more excuses to avoid that change.
My point is that up until this last event I was cognizant of the fact that the prices are too high, but was ok to pay for the quality I got, would have laughed in the face to anyone saying LTE coverage or speed are bad.
As a side note about the outage - I think a lot of people underestimate incompetence in large companies. I often marvel at how for example large Canadian banks are making record profits with many disfuctioning teams in critical positions - a situation that got much worse with the wave of the great resignation. If I were to bet, I’d say some incompetent team broke the infra, so there’s no need to look for conspiracy theories, people are just not competent in their positions and make mistakes when something has to be changed and they have no more excuses to avoid that change.
Worked for a large corp for many years. Lots of incompetence. If middle mgmt were properly incentivized they could probably have reduced head count by half or more. But if all large companies did this there would be mass unemployment.
Not surprised to see Rogers blow up. Looks good on them.
Not surprised to see Rogers blow up. Looks good on them.
TekSavvy is barely $5 cheaper these days for a co-ax connection. Not that I want to touch the Rogers network ever again if I have a choice. Not only are they incompetent, but coaxial copper is jittery, slow, and just overall awful. I want fibre.
It’s 2022 and I can’t get a fibre to the home connection in Ottawa less than 2km from Parliament. I’d get on a ladder myself to string up the fibre line myself if they’d let me.
Every time I want to self flagellate I take a look at the Swiss ISP init7[1]. 25/25Gbps to your door for 777 CHF/year. Infuriating that we can’t have that for any price here anywhere.
1: https://www.init7.net/en/internet/fiber7/
It’s 2022 and I can’t get a fibre to the home connection in Ottawa less than 2km from Parliament. I’d get on a ladder myself to string up the fibre line myself if they’d let me.
Every time I want to self flagellate I take a look at the Swiss ISP init7[1]. 25/25Gbps to your door for 777 CHF/year. Infuriating that we can’t have that for any price here anywhere.
1: https://www.init7.net/en/internet/fiber7/
The problem, as others pointed out, is regulatory capture. The market for telecommunications both in Canada and the US is entirely, completely corrupt, organized by unelected officials for the benefit of the few, at the expense of everyone in society.
It's as good an example as any of the lack of real liberty that entrepreneurs have in certain industries.
See also: banking and payments.
It's as good an example as any of the lack of real liberty that entrepreneurs have in certain industries.
See also: banking and payments.
> I am appalled that 911 services weren't even working in some cases.
This was the most surprising issue of the downtime. Rogers had just enough layers lit up for 9-1-1 calls to be attempted (to only fail) on its completely dark network that cell phones didn't/wouldn't failover that 9-1-1 call to the other mobile networks that were up.
This was the most surprising issue of the downtime. Rogers had just enough layers lit up for 9-1-1 calls to be attempted (to only fail) on its completely dark network that cell phones didn't/wouldn't failover that 9-1-1 call to the other mobile networks that were up.
I'm not sure why people are surprised about this.
It's all network traffic, and the network was DOWN. (It's not like when we had physical circuits, and if one part wasn't working it could be offlined without affecting the others).
911 traffic would have the highest priority, but without actual traffic flowing it's a ticket to nowhere. I'm guessing the cell towers had power, and likely the call routing software is programmed to drop calls to handle 911 requests, but still had no network to send the calls through.
As a user, I think you can disable your carrier (or, worst comes to worse, pop out your SIM card) to get picked up by another network -- but that isn't most people's thought process in an emergency.
Would be nice to see if this changes the approach at all -- example, If no network -> don't advertise or talk to our phones -> allows them to possibly roam or use other nets
It's all network traffic, and the network was DOWN. (It's not like when we had physical circuits, and if one part wasn't working it could be offlined without affecting the others).
911 traffic would have the highest priority, but without actual traffic flowing it's a ticket to nowhere. I'm guessing the cell towers had power, and likely the call routing software is programmed to drop calls to handle 911 requests, but still had no network to send the calls through.
As a user, I think you can disable your carrier (or, worst comes to worse, pop out your SIM card) to get picked up by another network -- but that isn't most people's thought process in an emergency.
Would be nice to see if this changes the approach at all -- example, If no network -> don't advertise or talk to our phones -> allows them to possibly roam or use other nets
I tried network selection, but could not select any other networks. Sadly no mutual aid deal was made to permit Rogers customers to connect to non-Rogers networks.
I suspect if you popped the SIM out, it would try any of the 3 providers available (in Toronto anyway), and dunno how the device chooses which. If it chose Rogers (strongest signal?), it would also fail to go through and you might be unable to get a 9-1-1 call in until you run in the right direction.
But few carry around a 'trombone' for emergencies, and I'd need something sharp to get my case off.
But my blame is on the cellular phone: A 9-1-1 call that makes it to the tower and no further shouldn't be counted as a success by the phone.
I suspect if you popped the SIM out, it would try any of the 3 providers available (in Toronto anyway), and dunno how the device chooses which. If it chose Rogers (strongest signal?), it would also fail to go through and you might be unable to get a 9-1-1 call in until you run in the right direction.
But few carry around a 'trombone' for emergencies, and I'd need something sharp to get my case off.
But my blame is on the cellular phone: A 9-1-1 call that makes it to the tower and no further shouldn't be counted as a success by the phone.
Telecommunication industry is heavily regulated, both in terms of Spectrum and Services requirement. So practically speaking the only way to improve Teleco is either you have government support to open up the market for competition, i.e Politics, or you have another iPhone moment ( the odds of happening is infinitely close to zero ) where a vendor could dictate Teleco for Network improvement.
I keep thinking the only way to break this anywhere in the world would be something like WiFi as a Mobile Network. Although this idea has been tried but never succeeded, practically because WiFi wasn't designed for such use in the first place. But recent improvement in WiFi 6 and WiFI 7 ( 802.11ax or 802.11be ) could solve this.
I keep thinking the only way to break this anywhere in the world would be something like WiFi as a Mobile Network. Although this idea has been tried but never succeeded, practically because WiFi wasn't designed for such use in the first place. But recent improvement in WiFi 6 and WiFI 7 ( 802.11ax or 802.11be ) could solve this.
"WiFi as mobile network" was sort of the concept behind WiMax, which was fairly aggressively trialed under a Clearwire/Comcast partnership in the US and didn't really go anywhere. Part of the disappearance of WiMax as as technology is simply that LTE won out as the 4G cellular standard rather than WiMax‚ but that came about in good part because WiMax was more suitable for fixed-point access than LTE and there was, and continues to be, little market interest in fixed-point wireless. Fundamentally it is extremely difficult for a wireless technology to compete with physical cable for fixed-point access, DOCSIS is pretty consistently keeping an order of magnitude ahead of the best wireless options.
As a curious bystander, why is wifi typically so short range? Today the frequencies of wifi and mobile networks seem to get increasingly similar, somewhere in single digit GHz. Yet my wifi router struggles with some drywall 3m away, despite several bulky antennas.
There's a few reasons:
1. WiFi is largely unmanaged so all the clients basically act like they're on an Ethernet hub. If they detect a signal they wait a random period before trying to send again.
2. Besides interference from nodes on the same network, you're getting interference from other networks on the same channel. Your WAP and WiFi card ignore those networks logically but they still need to yell at each other over the noise. This makes for dropped frames that need to be retransmitted.
3. WiFi is using unlicensed bands and has very low radiated power limits. The 2.4GHz and 5GHz bands in particular don't propagate well. This keeps a neighborhood of networks from making the spectrum totally unusable. Even still areas dense with WiFi networks make for really shitty interference.
Cellular on the other hand is aggressively managed. For one handsets and towers transmit on different channels. Your upstream won't affect my downstream. Besides every channel being multiplexed a tower can tell a handset to change channels or even entire bands in order to managed congestion.
1. WiFi is largely unmanaged so all the clients basically act like they're on an Ethernet hub. If they detect a signal they wait a random period before trying to send again.
2. Besides interference from nodes on the same network, you're getting interference from other networks on the same channel. Your WAP and WiFi card ignore those networks logically but they still need to yell at each other over the noise. This makes for dropped frames that need to be retransmitted.
3. WiFi is using unlicensed bands and has very low radiated power limits. The 2.4GHz and 5GHz bands in particular don't propagate well. This keeps a neighborhood of networks from making the spectrum totally unusable. Even still areas dense with WiFi networks make for really shitty interference.
Cellular on the other hand is aggressively managed. For one handsets and towers transmit on different channels. Your upstream won't affect my downstream. Besides every channel being multiplexed a tower can tell a handset to change channels or even entire bands in order to managed congestion.
Thank you! This was exactly the level of explanation I was looking for.
I believe consumer unregulated devices like wifi APs are limited to a broadcast power of 1W for one thing. Second, if you are indoors, it is more likely your phone is using a frequency band bellow 1Ghz
Different applications. Wi-Fi is designed to send bulk data but sacrifice coverage, while wireless telecommunications are designed to be reliable in thousands and have guaranteed coverage but sacrifices raw throughput.
It's actually very easy to start a carrier in Canada from a regulatory perspective. The difficulty is in achieving scale and profitability while dealing with the incumbents trying to squash you like a bug.
Starlink could also provide new backbone
No. Starlink does not have enough capacity for that.
For remote areas I would think it could? If the choice is between Starling and absolutely nothing, the former can provide at least some level of service. You'd also expect such remote areas to not be so densely populated so a single Starlink backhaul can be enough.
Could you elaborate a bit? Not sure why the parent comment was voted down, I'm genuinely curious
RF just simply does not have the capacity of fibre. Capacity upgrades in a satellite network take 5-10 years. Lighting up another wavelength on a glass fibre takes a few minutes to plug in new equipment at either end of the link.
Right now I see usage across my network at around 4-5 Mbps per subscriber at peak. It takes a few hundred users at those rates to overload a Starlink terminal with a gigabit port. Starlink is not a viable backhaul technology for anything but the smallest of networks.
Right now I see usage across my network at around 4-5 Mbps per subscriber at peak. It takes a few hundred users at those rates to overload a Starlink terminal with a gigabit port. Starlink is not a viable backhaul technology for anything but the smallest of networks.
For voice only?
Until recently I worked for a major Telco/VOIP carrier in Canada.
- ILEC/CLEC access - Spectrum access - HAH! https://www.canada.ca/en/innovation-science-economic-develop... it's a bit more than pocket change. - VOIP depends on ILECS
I could go on and go, but suffice to say if you don't have access to 5 million dollars for initial startup costs and then a lot more this is not something you want to deal with.
I'm on Rogers too so I get it. But there's a reason there's only a handful of ILEC in Canada. There's also a reason your cell phone only has access to a few providers nation wide.
The ss7 network is pretty damn rock solid - usually.
This seems to have been a failure at all levels: - SS7 - Network/BGP - Wireline - Wireless
This is unheard of in the telecom industry. Telecom is one of the most regulated industries in Canada - for good reason. We had teams of people just dealing with regulatory compliance issues.
Also to note 911 service is generally outsourced from the CLEC/ILEC to a couple specialized companies in Canada, it's not done inhouse.
It's the same with email. There's no email provider that has 100.0000% uptime.
I understand your outrage and I'm more than welcome to provide more details but this is not something that just happens overnight - or in 10 years.
You're obviously good with software, technology and all that stuff. Unfortunately, telecom/communication related industry is the exact opposite. It's an entirely different world, and even if you built the best shiny toys for endusers - it still has to interface with the old technology. We had lots of equipment from the 80s still running.
If you want to protect yourself in the future - use a good (non discount) voip provider in Canada who is an CLEC atleast themselves. There's a couple dozen nationwide. Most voip providers are resellers and uplink into a CLEC who then crossconnects with ILECs. The cheaper voip you get, the longer down the line you are and more things to go wrong.
Wow. Long ramble but hopefully I got my point across.
- ILEC/CLEC access - Spectrum access - HAH! https://www.canada.ca/en/innovation-science-economic-develop... it's a bit more than pocket change. - VOIP depends on ILECS
I could go on and go, but suffice to say if you don't have access to 5 million dollars for initial startup costs and then a lot more this is not something you want to deal with.
I'm on Rogers too so I get it. But there's a reason there's only a handful of ILEC in Canada. There's also a reason your cell phone only has access to a few providers nation wide.
The ss7 network is pretty damn rock solid - usually.
This seems to have been a failure at all levels: - SS7 - Network/BGP - Wireline - Wireless
This is unheard of in the telecom industry. Telecom is one of the most regulated industries in Canada - for good reason. We had teams of people just dealing with regulatory compliance issues.
Also to note 911 service is generally outsourced from the CLEC/ILEC to a couple specialized companies in Canada, it's not done inhouse.
It's the same with email. There's no email provider that has 100.0000% uptime.
I understand your outrage and I'm more than welcome to provide more details but this is not something that just happens overnight - or in 10 years.
You're obviously good with software, technology and all that stuff. Unfortunately, telecom/communication related industry is the exact opposite. It's an entirely different world, and even if you built the best shiny toys for endusers - it still has to interface with the old technology. We had lots of equipment from the 80s still running.
If you want to protect yourself in the future - use a good (non discount) voip provider in Canada who is an CLEC atleast themselves. There's a couple dozen nationwide. Most voip providers are resellers and uplink into a CLEC who then crossconnects with ILECs. The cheaper voip you get, the longer down the line you are and more things to go wrong.
Wow. Long ramble but hopefully I got my point across.
Northwestel (the telco that provides service to ALL of Northern Canada) had a similar outage in about 2013. It was utterly brutal.
They even stationed Police at street corners in Whitehorse and yellowknife in case anyone needed to ask for an ambulance or 911 assistance.
So it's not at all unprecedented.
They even stationed Police at street corners in Whitehorse and yellowknife in case anyone needed to ask for an ambulance or 911 assistance.
So it's not at all unprecedented.
Attorneys are probably the place to start a telcom.
A telcom typically requires access to public resources like right of ways and airwaves, and hence their are regulatory requirements to obtaining entitlement.
And there are contracts for interconnection with other network operators.
Contracts for vertical and conventional real-estate — so you need another stable of attorneys for that, too.
Which means you need pockets so deep that cost doesn’t matter…e.g. a government contract.
Which means you need a government that wants you to do what you are trying to do.
Otherwise you won’t get the avigation easements and similar things your towers and wires need.
Good luck.
A telcom typically requires access to public resources like right of ways and airwaves, and hence their are regulatory requirements to obtaining entitlement.
And there are contracts for interconnection with other network operators.
Contracts for vertical and conventional real-estate — so you need another stable of attorneys for that, too.
Which means you need pockets so deep that cost doesn’t matter…e.g. a government contract.
Which means you need a government that wants you to do what you are trying to do.
Otherwise you won’t get the avigation easements and similar things your towers and wires need.
Good luck.
This, in my opinion, is the most important comment in this thread.
I would also add that since you are forming a company that is considered a public utility, you are also mandated [in the US] to create a tariff to list the rates and terms at which you’ll sell service.
Then be prepared for the incumbent (or a nationwide carrier) to order items off of your tariff to reach your customers with their services over your physical facilities.
I would also add that since you are forming a company that is considered a public utility, you are also mandated [in the US] to create a tariff to list the rates and terms at which you’ll sell service.
Then be prepared for the incumbent (or a nationwide carrier) to order items off of your tariff to reach your customers with their services over your physical facilities.
Buy enough politicians to capture the ability to own and operate a monopoly telco run as a coop/utility. Put engineering talent in leadership roles, setting the necessary culture. Everything else is access to capital and effective execution of physical infra buildout and O&M (“the easy part”).
Any country but Canada, the incumbents there have so much political cover and ways to make your life miserable that it likely will not work. I share your pain, and have tried to get a small ISP off the ground in rural community after getting the runaround from the usual suspects about broadband service, after acquiring all the gear and signing up the first customers instantly we had fiber available (when before that wasn't in the planning for the next decade or so). But best of luck regardless.
An old friend of mine setup an internet company in Canada. They used point to point wireless, and in some cases exchanged free Internet for being allowed to use people's high utility poles / structures, etc to beam wireless. This was for rural canada, so a bit specific though.
https://www.bbc.com/news/technology-37974267
The cost of physical infrastructure deployment can shift significantly based on accessible good will. Purchasing space, requiring local permits and paying full price for equipment and person-power drastically alter margins.
It significantly depends on the deployment goals you have. If you can find deals to share infrastructure and deployment cost with other organizations (such as a power company) that can work well too, but in larger countries this kind of deal is hard won and more often fails than succeeds.
Wireless is much cheaper than cables, but has a high ongoing maintenance cost, particularly if it is deployed nearby or directly competing with other signals in the unlicensed bands. Wireless also always ends up competing with some amount of crackpot pushback for major deployments - that is not to diminish those with genuine measured concern, but there is often a vocal group that show up who represent strong positions way beyond any reviewed evidence leading to further challenge in the political space.
The cost of physical infrastructure deployment can shift significantly based on accessible good will. Purchasing space, requiring local permits and paying full price for equipment and person-power drastically alter margins.
It significantly depends on the deployment goals you have. If you can find deals to share infrastructure and deployment cost with other organizations (such as a power company) that can work well too, but in larger countries this kind of deal is hard won and more often fails than succeeds.
Wireless is much cheaper than cables, but has a high ongoing maintenance cost, particularly if it is deployed nearby or directly competing with other signals in the unlicensed bands. Wireless also always ends up competing with some amount of crackpot pushback for major deployments - that is not to diminish those with genuine measured concern, but there is often a vocal group that show up who represent strong positions way beyond any reviewed evidence leading to further challenge in the political space.
Already lots of advice, but I'll chime in too.
Telcos are actually a few parts -- there is Voice, which can be landlines or cellular, there is Data which is usually Internet but can be leased lines. Modern networks do allow a lot of overlap with these, but each of them have some bits that are very specialized and need equipment/people.
VoIP and Internet is a decent place to start. The "core" is fairly easy to do, the harder part is the "last mile". Provinces and Municipalities usually charge to allow you to use their poles or right of ways.
The most interesting idea I've heard is building a non-profit last mile with the help of your municipality. At one point Toronto Hydro had an extensive fiber network, and it would have been easy to do; but I think the city sold off Hydro and Hydro in turn sold off their fiber network.
I've also heard of getting a neighborhood together to buy a commercial tier connection, and then distributing it, but there are some details that need to be worked out to make sure this is workable.
Telcos are actually a few parts -- there is Voice, which can be landlines or cellular, there is Data which is usually Internet but can be leased lines. Modern networks do allow a lot of overlap with these, but each of them have some bits that are very specialized and need equipment/people.
VoIP and Internet is a decent place to start. The "core" is fairly easy to do, the harder part is the "last mile". Provinces and Municipalities usually charge to allow you to use their poles or right of ways.
The most interesting idea I've heard is building a non-profit last mile with the help of your municipality. At one point Toronto Hydro had an extensive fiber network, and it would have been easy to do; but I think the city sold off Hydro and Hydro in turn sold off their fiber network.
I've also heard of getting a neighborhood together to buy a commercial tier connection, and then distributing it, but there are some details that need to be worked out to make sure this is workable.
I would look hard at the entire history of Tucows' efforts with Ting Mobile and Ting Internet. Lots of important lessons there.
Wind (Freedom) wasn't simply acquired. It was hampered and restricted by existing Canadian laws to do with foreign ownership.
> doesn't create any redundant networks.
I would say that redundant networks are precisely whats required.
Selling a government on the idea is the hard part. Actually maintaining communications links, wires or fiber etc, is apparently so trivial a challenge as to be a minority of most phone companies actual business. They spend much more effort on billing and marketing.
I would say that redundant networks are precisely whats required.
Selling a government on the idea is the hard part. Actually maintaining communications links, wires or fiber etc, is apparently so trivial a challenge as to be a minority of most phone companies actual business. They spend much more effort on billing and marketing.
I can't reply direct for some reason, but wanted to comment on...
> But there's a reason there's only a handful of ILEC in Canada
(ILEC for those unaware are Incumbent local exchange carrier, as opposed to CLECs or Competitive local exchange carrier, once local competition was introduced) Yes, because when the Phone networks were started, it was SUPER expensive to wire them up, so the government gave MONOPOLIES away for REGULATED returns.
So, for the most part, each geographic area had one (or one main) phone company. And a few of those companies merged or were bought out as competition entered the space, leaving less than a handful.
It was felt that balancing access for exclusivity was a good trade, and it probably was. But now, "copper" landline service is strictly regulated, while Internet and Cellular are unregulated. (This is why, despite landline being super cheap to run, it costs MORE then entry cellphone plans per month).
> But there's a reason there's only a handful of ILEC in Canada
(ILEC for those unaware are Incumbent local exchange carrier, as opposed to CLECs or Competitive local exchange carrier, once local competition was introduced) Yes, because when the Phone networks were started, it was SUPER expensive to wire them up, so the government gave MONOPOLIES away for REGULATED returns.
So, for the most part, each geographic area had one (or one main) phone company. And a few of those companies merged or were bought out as competition entered the space, leaving less than a handful.
It was felt that balancing access for exclusivity was a good trade, and it probably was. But now, "copper" landline service is strictly regulated, while Internet and Cellular are unregulated. (This is why, despite landline being super cheap to run, it costs MORE then entry cellphone plans per month).
Just wanna add one thing I've not seen below: telco is heavily regulated, so you definitely need to get a lawyer to either guide you thru the minefield, eg know exactly what phrases and service offerings stay on the less-regulated side of the law and which dont. Not all regs are bad (eg anyone can call 911, anywhere, and the network must figure out where they are) but its not for the faint hearted, and some might argue that the big telcos like it like that.
Also, if you are going to do anything with voice, public cloud may not cut it. The reliability bar for voice services is insanely high. Sub 10ms failover between elements while preserving the voice media stream type of stuff. Five or six nines availability, or else the regulator beats the shi* out of you. This is not an outage model where your COO can put out a chatty tweet saying "sorry and heres a $10 credit."
Also, if you are going to do anything with voice, public cloud may not cut it. The reliability bar for voice services is insanely high. Sub 10ms failover between elements while preserving the voice media stream type of stuff. Five or six nines availability, or else the regulator beats the shi* out of you. This is not an outage model where your COO can put out a chatty tweet saying "sorry and heres a $10 credit."
At least in Canada, you can't afford the lawyers that have regulatory experience and are good enough to win. They're already employed by the large ILECs. Or they don't know how to strategize and win at the 5d chess games the incumbents are playing to win for the long term. TekSavvy and CNOC played right into the goals of Bell with disaggregated. They tried to address their number one cost of capacity charges on last mile aggregated DSL networks (which were north of $10 per megabit) and asked for disaggregated to eliminate that cost. Bell saw that as a oppourtunity to load a whole bunch of costs into provisioning of disaggregated and ended up with several orders of magnitude higher costs for wholesale ISPs. Aggregated setup costs were about $10-15k for all of Ontario and Quebec. Disaggregated costs $50-150k per CO and would cost hundreds of millions to setup across all of Ontario and Quebec. Since FTTH is tied to disaggregated, Teksavvy has basically lost a large chunk of its market by backing the wrong horse.
There are maybe a few dozen people across Canada that truly understand what's going on in the Canadian telecom regulatory world. More people need to educate themselves about what's going on and ask the CRTC to improve the major pain points holding back independent providers.
There are maybe a few dozen people across Canada that truly understand what's going on in the Canadian telecom regulatory world. More people need to educate themselves about what's going on and ask the CRTC to improve the major pain points holding back independent providers.
To your last paragraph, how does one adequately educate oneself to advocate effectively at the CRTC if the ranks of those truly in the know are so small and inaccessible (seemingly by design)?
It's not by design so much as a result of limited participation by the general public. Proceedings are posted to the CRTC's website and are available for anyone to view and comment on.
When it comes to information that the CRTC considers in its decisions, as the CRTC functions quite similarly to a court: they will not consider evidence that is not presented to its version of the public record. Unless the general public submits their comments to proceedings, the comments tend to be limited to the views of carriers and a few public interest advocacy bodies. Stories published in the media, on blogs or elsewhere will not be considered unless someone submits them to the CRTC's record, even though they may be readily access to the CRTC.
This becomes quite important when there is locally relevant information that the commission is not aware of, or if there are business practices that the commission should be made aware of. People need to complain when their choice of internet providers is limited or if incumbents increase prices twice a year. If nobody raises these issues at the CRTC, nothing will change.
When it comes to information that the CRTC considers in its decisions, as the CRTC functions quite similarly to a court: they will not consider evidence that is not presented to its version of the public record. Unless the general public submits their comments to proceedings, the comments tend to be limited to the views of carriers and a few public interest advocacy bodies. Stories published in the media, on blogs or elsewhere will not be considered unless someone submits them to the CRTC's record, even though they may be readily access to the CRTC.
This becomes quite important when there is locally relevant information that the commission is not aware of, or if there are business practices that the commission should be made aware of. People need to complain when their choice of internet providers is limited or if incumbents increase prices twice a year. If nobody raises these issues at the CRTC, nothing will change.
And if you are going to do anything with voice, you are mandated to connect to the ILEC’s interconnection points (also known as tandems) for exchanging local traffic (as well as negotiating a new interconnection agreement or opting into an existing one).
You are also required to connect to all of your region’s 911 interconnection points (also known as public safety access points, abbreviated as PSAPs) which are distinctly different than the aforementioned interconnection points.
Finally, if you have an outage over 30 minutes in the US, in most cases, you are required to report that to the FCC within 2 hours of the event, then submit a final post-mortem within 30 days.
Voice is no joke.
You are also required to connect to all of your region’s 911 interconnection points (also known as public safety access points, abbreviated as PSAPs) which are distinctly different than the aforementioned interconnection points.
Finally, if you have an outage over 30 minutes in the US, in most cases, you are required to report that to the FCC within 2 hours of the event, then submit a final post-mortem within 30 days.
Voice is no joke.
> Also, if you are going to do anything with voice, public cloud may not cut it.
Non sense. I run a very successful over-the-top VOIP provider using commodity dedicated servers in multiple states, some even on Digital Ocean. Essentially no issues related to connectivity, be it business customers or residentials.
Non sense. I run a very successful over-the-top VOIP provider using commodity dedicated servers in multiple states, some even on Digital Ocean. Essentially no issues related to connectivity, be it business customers or residentials.
Interesting! Do you process RTP media in your instances? If so, and you can share, how do you do fast failover between instances without loss of the media stream?
Im not familiar with DO but on the big 3, none of them can do IP address failover via L2 GARP because of how their fabric works, and doing it at L3 with say BGP is waaaay too slow.
Im not familiar with DO but on the big 3, none of them can do IP address failover via L2 GARP because of how their fabric works, and doing it at L3 with say BGP is waaaay too slow.
> Five or six nines availability
Post is about telecom service that was down for a whole day
Post is about telecom service that was down for a whole day
Take a look at https://www.helium.com/
They have nascent 5G support, and it’s all “owned by the people”.
Not a big fan of crypto shenanigans, but Helium comes as close as possible to crypto that kinda works and solves a problem.
They have nascent 5G support, and it’s all “owned by the people”.
Not a big fan of crypto shenanigans, but Helium comes as close as possible to crypto that kinda works and solves a problem.
I’d sooner sign a 10 year contract with Bell than get in on a crypto backed scheme for something as critical as my internet connectivity. Devil you know, etc. Absolute non-starter.
Besides, there are actually successful mesh nets already out there in the wild, like Freifunk in Berlin.
https://berlin.freifunk.net/index_en/
Besides, there are actually successful mesh nets already out there in the wild, like Freifunk in Berlin.
https://berlin.freifunk.net/index_en/
Helium is the only serious attempt to solve the bootstrapping problem and has demonstrated some significant early traction. The fact that there's only one mention among all these comments is really unfortunate.
https://www.forbes.com/forbes/2006/0410/063.html?sh=85b4f7b4...
The subtitle is "Want to build a phone company for $100? Give Mark Spencer a ring?"
YMMV because this article was published in 2006 and Digium, the parent co was sold in 2018. But the thinking about the process is still relevant and there is a book on Asterisk on Amazon....
HTH
The subtitle is "Want to build a phone company for $100? Give Mark Spencer a ring?"
YMMV because this article was published in 2006 and Digium, the parent co was sold in 2018. But the thinking about the process is still relevant and there is a book on Asterisk on Amazon....
HTH
While you’re building the business try getting your municipality to subsidize access to poles and existing infrastructure.
Join with other providers like Teksavvy to help the CRTC to break up the monopoly on the infrastructure.
Join with other providers like Teksavvy to help the CRTC to break up the monopoly on the infrastructure.
I think there's an opportunity to leverage Starlink's concept at a municipal/regional level. Maybe Starlink will do this to sell into denser regions with miniaturized equipment. Get a high enough vantage point for the remote end, on the CPE side, issue steerable array antennas that can be self-installed.
As demand increases, you could setup more remote ends and enough steerable array antennas can automatically re-aim and new subs will point to the new one.
As demand increases, you could setup more remote ends and enough steerable array antennas can automatically re-aim and new subs will point to the new one.
In Canada? You can't.
Canada doesn't have a lack of competition. Canada has a lack of people willing to consider competition.
https://www.youtube.com/watch?v=HlRPLBLkZK0
Everyone I know whines about high telecom and internet prices. I know very few people willing to consider anything other than a lower price at Bell, Rogers, or Telus.
Canadians are very risk averse people.
Canada doesn't have a lack of competition. Canada has a lack of people willing to consider competition.
https://www.youtube.com/watch?v=HlRPLBLkZK0
Everyone I know whines about high telecom and internet prices. I know very few people willing to consider anything other than a lower price at Bell, Rogers, or Telus.
Canadians are very risk averse people.
[deleted]
Wind was interesting in that they got started, afaik, by operating on different bands entirely that didn't have the reach of any others, and then offered ostensibly better plans on that spectrum. It was kind of rough at the start though, and you were basically roaming outside of a few areas. Also, only a few phones at the time had the right radios.
There are plenty of wireless micro-ISPs in the Bay Area. Our small office used a gigabit wireless service to augment xfinity, my sibling has a wireless via Sail, there's Ridge Wireless, Wave, and so on.
It would probably be worth talking to one of these to find out what the real experience is like. In my experience, people will gladly talk about what they've learned.
It would probably be worth talking to one of these to find out what the real experience is like. In my experience, people will gladly talk about what they've learned.
Canada is huge and sparsely populated. There's a reason that they can only muster a few places above the border to settle. Pick a community just outside of a metro and setup shop. You need density and you might as well play city planning to some advantage. Wisp may be the fastest way to start. Business customers will be your lifeblood.
You need new physical data routes, which isn't cheap. I'd start working on a local non-profit, low bandwidth effort first. Something that can be used in an emergency to route around damage.
Also, see about using StarLink.
Also, do you have ComCast in addition to the phone company? It's always good to have a backup route to the world.
Also, see about using StarLink.
Also, do you have ComCast in addition to the phone company? It's always good to have a backup route to the world.
Yesterday's outage was not an L0/L1 problem, it was L2 and L3. Rogers wavelength services continued working perfectly fine (I have 3 x 10Gbps circuits from them). Even the Rogers owned data center my gear is in was fine as they were an acquisition that still has IP connectivity from other carriers (including Zayo).
As the owner and operator of a small non-dominant carrier, what would really help is to have access to fibre local loops and transport at reasonable costs that reflect their 30+ year operational life. Existing fibre owned by the incumbents doesn't need to be duplicated. Unfortunately that's not the path the CRTC has taken to encourage competition. I really need to write a new Part 1 application on this.
As the owner and operator of a small non-dominant carrier, what would really help is to have access to fibre local loops and transport at reasonable costs that reflect their 30+ year operational life. Existing fibre owned by the incumbents doesn't need to be duplicated. Unfortunately that's not the path the CRTC has taken to encourage competition. I really need to write a new Part 1 application on this.
Please do. And good luck with that. The level of national frustration with telecoms must have reached a new record yesterday.
If I had infinite time, I would. It's taken a few years to recover from no fewer than 4 projects which were complete write offs in terms of profitability because my permits were delayed while the incumbent built out a neighbourhood for themselves. There's nothing better than a half million dollar hole in your financial plans thanks to delayed and denied permits.
Starting to build a carrier is a really steep learning curve. You don't know what the rules are, and you don't know how and when to fight back against the incumbents. Hell, the CRTC is a huge problem in and of itself as it took 2 years to sort out a fairly simple filing by Videotron where they had permits improperly denied to access poles (see https://crtc.gc.ca/eng/archive/2022/2022-160.htm for the decision on that filing -- my comments were referenced several times in that decision).
If someone really wants to start a small telco here in Canada, my number 1 recommendation is: do your own engineering. Engineering consulting companies will dig you a very expensive hole for projects that go sideways -- avoid that. The second most important tip is to fight back and go to regulators the moment permits do not get issued in a timely manner. Learn from some of my pain to avoid your own expensive lessons learned.
Starting to build a carrier is a really steep learning curve. You don't know what the rules are, and you don't know how and when to fight back against the incumbents. Hell, the CRTC is a huge problem in and of itself as it took 2 years to sort out a fairly simple filing by Videotron where they had permits improperly denied to access poles (see https://crtc.gc.ca/eng/archive/2022/2022-160.htm for the decision on that filing -- my comments were referenced several times in that decision).
If someone really wants to start a small telco here in Canada, my number 1 recommendation is: do your own engineering. Engineering consulting companies will dig you a very expensive hole for projects that go sideways -- avoid that. The second most important tip is to fight back and go to regulators the moment permits do not get issued in a timely manner. Learn from some of my pain to avoid your own expensive lessons learned.
> Also, do you have ComCast in addition to the phone company? It's always good to have a backup route to the world.
While a reasonable comment in the US, in most cases the phone company and the cable company in Canada are usually one and the same or have a common parent (and if the Rogers-Shaw merger is closed would make this true to nearly 90% of Canadians).
While a reasonable comment in the US, in most cases the phone company and the cable company in Canada are usually one and the same or have a common parent (and if the Rogers-Shaw merger is closed would make this true to nearly 90% of Canadians).
Armchair ISP bootstrapper comment: try to start in one location. And fuck permits, go full pirate on these assholes. Become popular for providing exceptional though very limited service in a small geographic area. Then social media campaigning to paint the incumbents black. Try to parlay from there.
Can’t do this in my building (NYC) due to line-of-sight issues, but worth investigating this:
https://www.nycmesh.net/
https://www.nycmesh.net/faq
https://www.nycmesh.net/
https://www.nycmesh.net/faq
If you were having these outages on a regular basis then I'd sympathise more. All the expense and trouble of starting a rival telco because of a single outage of the incumbent(s) sounds a bit knee-jerk and extreme...
Some random thoughts:
1. You didn't specify wireless or wired. It matters. With wired, your biggest problem is going to be getting permission to deploy towers and licensing part of the spectrum;
2. Wired avoids this but adds the problem (and the biggest cost) of the last mile. Where do you run those cables?
a) Poles: you need permission for this and there can be a bunch of hyperlocal laws that will get in your way. For example, you may have to wait for existing telcos to move their wires up or down to make room and they'll take their sweet time.
b) In trenches: you'll need a different kind of permission for this. Local conditions may make this more of less difficult (eg there can be alot of large rocks in the soil).
3. If you run a cable from a house and on a pole or in a trench where does it go? You need some kinf of substation for this depending on the max transmission length of the cables you employ. You need planning permission for that and teh big cost of the real estate required. If it's a hole in the ground, that's less of a problem. If it's a free standing structure containing switching equipment, expect residents to fight it on NIMBY grounds;
4. Depending on where you are you may face local or state laws that restrict your ability to do any of this. Some US states have passed laws against municipal broadband (at the behest of telco lobby). Telcos may have exclusive franchises with certain areas. This has been dismantled somewhat but you may find unofficial opposition anyway;
5. You're going to need installer for all of this and people to handle all the permitting;
6. Once it's built you need people to maintain it. People will come through and cut your wires or they'll be accidents like cheery pickers running through your cables or poles going down in a storm requiring cable repair.
7. Existing telcos are exceptionally good at playing this game and are typically in bed with local politicians who will make your life difficult.
8. Will you get enough customers? Just getting permission to build something will probably lead existing telcos to cut prices and lock in consumers with 12 or 24 month contracts to starve you.
9. Do you have a TV service? If not, you're going to lose a certain number of potential customers who still want this. If you do, the price you pay scales with your size. Verizon, Comcat or AT&T pay a lot less to content providers for their TV packages than you will. Telcos like to bundle TV and Internet so you may find your customers only pay $20/month extra to add Internet to theri TV package.
10. Who do you get upstream Internet connections from? It may well be one of your local competitors. Guess how that goes.
This is a capital intensive business that doesn't reward overbuilds. The only way this can really work is with heavy cooperation from your local municpality. As mentioned there may be legal barries from them doing so (as noted, telcos don't want a repeat of Chattanooga).
This'll work best in low population centers that are underserved or not served at all by existing players. Low density housing means getting trenching permission is going to be less of a problem. Potential customers will also be much more motivated if there other options are HughestNET or DSL.
Starlink probably means even these customers aren't going to be as motivated as they would've been 5-10 years ago.
In more rural areas you may even have the option of microwave relays that'll bypass a spectrum issues and it'll be easier to permission to be a directional wireless tower on someone's boundary.
1. You didn't specify wireless or wired. It matters. With wired, your biggest problem is going to be getting permission to deploy towers and licensing part of the spectrum;
2. Wired avoids this but adds the problem (and the biggest cost) of the last mile. Where do you run those cables?
a) Poles: you need permission for this and there can be a bunch of hyperlocal laws that will get in your way. For example, you may have to wait for existing telcos to move their wires up or down to make room and they'll take their sweet time.
b) In trenches: you'll need a different kind of permission for this. Local conditions may make this more of less difficult (eg there can be alot of large rocks in the soil).
3. If you run a cable from a house and on a pole or in a trench where does it go? You need some kinf of substation for this depending on the max transmission length of the cables you employ. You need planning permission for that and teh big cost of the real estate required. If it's a hole in the ground, that's less of a problem. If it's a free standing structure containing switching equipment, expect residents to fight it on NIMBY grounds;
4. Depending on where you are you may face local or state laws that restrict your ability to do any of this. Some US states have passed laws against municipal broadband (at the behest of telco lobby). Telcos may have exclusive franchises with certain areas. This has been dismantled somewhat but you may find unofficial opposition anyway;
5. You're going to need installer for all of this and people to handle all the permitting;
6. Once it's built you need people to maintain it. People will come through and cut your wires or they'll be accidents like cheery pickers running through your cables or poles going down in a storm requiring cable repair.
7. Existing telcos are exceptionally good at playing this game and are typically in bed with local politicians who will make your life difficult.
8. Will you get enough customers? Just getting permission to build something will probably lead existing telcos to cut prices and lock in consumers with 12 or 24 month contracts to starve you.
9. Do you have a TV service? If not, you're going to lose a certain number of potential customers who still want this. If you do, the price you pay scales with your size. Verizon, Comcat or AT&T pay a lot less to content providers for their TV packages than you will. Telcos like to bundle TV and Internet so you may find your customers only pay $20/month extra to add Internet to theri TV package.
10. Who do you get upstream Internet connections from? It may well be one of your local competitors. Guess how that goes.
This is a capital intensive business that doesn't reward overbuilds. The only way this can really work is with heavy cooperation from your local municpality. As mentioned there may be legal barries from them doing so (as noted, telcos don't want a repeat of Chattanooga).
This'll work best in low population centers that are underserved or not served at all by existing players. Low density housing means getting trenching permission is going to be less of a problem. Potential customers will also be much more motivated if there other options are HughestNET or DSL.
Starlink probably means even these customers aren't going to be as motivated as they would've been 5-10 years ago.
In more rural areas you may even have the option of microwave relays that'll bypass a spectrum issues and it'll be easier to permission to be a directional wireless tower on someone's boundary.
Happened in India recently. Read the Reliance Jio story.
[deleted]
Given the incumbents here, you might want to consider starting with dismantling capitalism over seeking market-driven solutions for cronies
Be careful ... its hard to become the embodiment of the thing you hate. Running an ISP can be rough on your reputation :)
So this question really depends on what the objective is. This really seems to be a knee jerk reaction to a big outage, and an overall dissatisfaction with the big players.
If the objective is to compete on a national scale, we're talking capital investments of something like $10 billion CDN per year. As I recall when I worked in the industry Bell was spending 5-6 billion per year on capital investments, and if you're starting now you're playing catch up, so probably need to double that for a decade or two to catch up.
And that's just capital investments, equipment, software, and cables in the ground. Can this be run way more efficiently, and with less overhead in billing systems and pet projects and the like, sure. But use these numbers just for the realm we need to be in. Let alone the people to build, maintain, upgrade, monitor, respond to outages, etc. Just to try and put that into perspective, the entire federal government spent about $440 billion last year. So we're talking a good 3-4% of the federal budget, for what one of the big 3 is spending. And keeping in mind, they have a lot of power to compete on price and features wherever you get started.
So I'm personally believe that difficult to duplicate infrastructure should be government run. It's simply impractical to go and say we'll have 5 or 10 or 15 competitors, and startups, and bankruptcies when the cables in the ground are so capital intensive. We don't run 15 power lines to each house so everyone can change on a whim. Why are we pretending we can run 15 fiber cables and do the same thing. We don't really want one new big competitor, we want a vibrant market of new approaches and failures to occur.
So set that policy tirade aside, and let's scope down our expectations from nationwide.
I get internet from a small provider, who mainly serves multi-dwelling buildings in the Toronto area. What they've been able to carve out, is a niche where it's cost effective to compete and start from. They're basically leveraging some, I think it was Industry Canada rules changes, that basically say that the telcos aren't able to claim ownership of the wiring inside a building. The provider I use, then just hired some Bell technicians, bought the same DSL remotes Bell was using for the same services, purchased a leased MPLS or similar line back to their data center to the basement of the building, and started making deals with Condo Boards and building owners to offer services. So an internet connection, DSL remote, and some technicians to terminate wires was what they needed to get started. Of course it's more complicated than this, but that's what it sort of boiled down to.
The DSL remotes have a SIP client that can create a phone connection, so you can get home phone, and they built a cheapo IPTV service with some android boxes. Give away telephone, TV and internet for free in the common areas of the building for advertising.
911 services, they just contacted with whoever provides normal 911 services for voip providers.
When I switched to them, it was cheaper than I got with an employee discount in the industry. Last year, they put fiber in, so I've got unlimited gigabit fiber for half the price or less of the big companies. The Big companies of course, now offer unpublished promotions from time to time that are only available to my building. This of course doesn't cover cellular service, but we're back to that capital intensive replication of infrastructure.
I'm glad I'm out of this industry.
If the objective is to compete on a national scale, we're talking capital investments of something like $10 billion CDN per year. As I recall when I worked in the industry Bell was spending 5-6 billion per year on capital investments, and if you're starting now you're playing catch up, so probably need to double that for a decade or two to catch up.
And that's just capital investments, equipment, software, and cables in the ground. Can this be run way more efficiently, and with less overhead in billing systems and pet projects and the like, sure. But use these numbers just for the realm we need to be in. Let alone the people to build, maintain, upgrade, monitor, respond to outages, etc. Just to try and put that into perspective, the entire federal government spent about $440 billion last year. So we're talking a good 3-4% of the federal budget, for what one of the big 3 is spending. And keeping in mind, they have a lot of power to compete on price and features wherever you get started.
So I'm personally believe that difficult to duplicate infrastructure should be government run. It's simply impractical to go and say we'll have 5 or 10 or 15 competitors, and startups, and bankruptcies when the cables in the ground are so capital intensive. We don't run 15 power lines to each house so everyone can change on a whim. Why are we pretending we can run 15 fiber cables and do the same thing. We don't really want one new big competitor, we want a vibrant market of new approaches and failures to occur.
So set that policy tirade aside, and let's scope down our expectations from nationwide.
I get internet from a small provider, who mainly serves multi-dwelling buildings in the Toronto area. What they've been able to carve out, is a niche where it's cost effective to compete and start from. They're basically leveraging some, I think it was Industry Canada rules changes, that basically say that the telcos aren't able to claim ownership of the wiring inside a building. The provider I use, then just hired some Bell technicians, bought the same DSL remotes Bell was using for the same services, purchased a leased MPLS or similar line back to their data center to the basement of the building, and started making deals with Condo Boards and building owners to offer services. So an internet connection, DSL remote, and some technicians to terminate wires was what they needed to get started. Of course it's more complicated than this, but that's what it sort of boiled down to.
The DSL remotes have a SIP client that can create a phone connection, so you can get home phone, and they built a cheapo IPTV service with some android boxes. Give away telephone, TV and internet for free in the common areas of the building for advertising.
911 services, they just contacted with whoever provides normal 911 services for voip providers.
When I switched to them, it was cheaper than I got with an employee discount in the industry. Last year, they put fiber in, so I've got unlimited gigabit fiber for half the price or less of the big companies. The Big companies of course, now offer unpublished promotions from time to time that are only available to my building. This of course doesn't cover cellular service, but we're back to that capital intensive replication of infrastructure.
I'm glad I'm out of this industry.
Note. There is no internet backbone. You pay to peer your traffic.
> You pay to peer your traffic.
That's not the whole truth.
At least here in Germany, there are publicly-ish peering points. You pay a small amount for colocating a single router in the data center, and each peering partner only accepts IP traffic for the IPs they feel responsible for (typically their own AS). Companies such as Google participate in those, so you can offload all of your Google and Youtube traffic at such a peering point. A small ISP can afford to participate in such a peering point, you only pay a fixed, monthly fee for colocation.
However, you likely want to also have a carrier to whom you can hand off all the other traffic that you don't have direct peers for.
(Source: I work for a company that hosts some of the POPs for https://www.peeringdb.com/ix/135)
That's not the whole truth.
At least here in Germany, there are publicly-ish peering points. You pay a small amount for colocating a single router in the data center, and each peering partner only accepts IP traffic for the IPs they feel responsible for (typically their own AS). Companies such as Google participate in those, so you can offload all of your Google and Youtube traffic at such a peering point. A small ISP can afford to participate in such a peering point, you only pay a fixed, monthly fee for colocation.
However, you likely want to also have a carrier to whom you can hand off all the other traffic that you don't have direct peers for.
(Source: I work for a company that hosts some of the POPs for https://www.peeringdb.com/ix/135)
A new startup ISP won't be peering, they'll be paying an incumbent provider for at least transit if not full access services (e.g. "dedicated internet access," basically commercial ISP service under contract terms that allow diverse uses including internet services). The term "backbone" is pretty useful here as the number of providers in North America with real global connectivity is fairly small; most ISPs with under thousands of customers are themselves customers of Centurylink/Lumen or Verizon, both of which have been steadily acquiring their competition in core internet connectivity (Level3 for example).
You don't need to buy DIA from an incumbent. You can get L1 / L2 circuits from various carriers back to data centres where you can buy transit and peer with other ISPs using BGP. L1 wavelength circuits did not suffer from the outage Rogers had yesterday, but the downside of wavelengths is that they require physical intervention to recover from failures.
For exchanges, Ontario has the Torix exchange in Toronto for peering, and there's QIX in Montreal. Ottawa has Ottix, but it does not have many peers. There are others out west (as I'm in Ontario and don't pay attention to them as much).
For exchanges, Ontario has the Torix exchange in Toronto for peering, and there's QIX in Montreal. Ottawa has Ottix, but it does not have many peers. There are others out west (as I'm in Ontario and don't pay attention to them as much).
Torix has most of the peers you need for a majority of an ISP's traffic.
Most consumer traffic is going to be to CDNs (Limelight, Akamai, Cloudflare), AWS, Apple, Google/Youtube, Microsoft, Netflix and the like if you're not big enough to have a private interconnect to them. It turns out the big internet companies like IX's too and don't want backbone providers being gatekeeper.
Most consumer traffic is going to be to CDNs (Limelight, Akamai, Cloudflare), AWS, Apple, Google/Youtube, Microsoft, Netflix and the like if you're not big enough to have a private interconnect to them. It turns out the big internet companies like IX's too and don't want backbone providers being gatekeeper.
There most certainly is an internet backbone. The backbone is the the group of Tier 1 providers in a specific country. Tier 1 providers have settlement-free peering with all the other Tier 1 providers in their country[1][2][3].
At any rate a new ISP wouldn't need to do paid peering they would simply buy transit from a Tier 1 or Tier 2 provider.
[1] https://drpeering.net/FAQ/Who-are-the-Tier-1-ISPs.php
[2] https://www.thousandeyes.com/learning/techtorials/isp-tiers
[3] https://drpeering.net/FAQ/What-is-a-Tier-1-ISP.php
At any rate a new ISP wouldn't need to do paid peering they would simply buy transit from a Tier 1 or Tier 2 provider.
[1] https://drpeering.net/FAQ/Who-are-the-Tier-1-ISPs.php
[2] https://www.thousandeyes.com/learning/techtorials/isp-tiers
[3] https://drpeering.net/FAQ/What-is-a-Tier-1-ISP.php
Start at one location in a time? Fellow Vancourite here as well
I'd make sure you have a damn good lawyer.
Absolutely no idea, to answer your question directly. The only though I had was maybe start with WISP and gain experience? Ubiquiti offers a stomach-able platform to get your off the ground.
Bootstrapping likely means outsized leveraging of knowledge with smaller amounts of cash focused on sustainability from the start.
Maybe there’s a fintech type disruption on the horizon.
This would be a fun project to hack on with 5-10 skilled folks with complementary skills to cover the industry, legal, financial, operational and the hardware and software angles to see what might be possible today. There have to be folks with a lot of experience in Canada that might be worth seeking out before beginning that feel exactly like you do and I would say I do too.
How much cash investment would you start with?
I’d be curious to how easy it can be for a non technical consumer today to access an mvno service that is turnkey and sells 3-4gb a month tablet type data plan (currently $10-20/mo) that can handle calls and sms via a carrier grade voip for $20-30/mo might be an interesting. It’s not new, but it’s not always easy to setup unless you’re technical. Calls are secondary to data (WhatsApp calls) these days for many. Keeping it independent of one carrier doesn’t preclude the fact that existing carriers have likely dealt with this type of approaches before. But it’s ok to start and remain small and let growth be organic snarfed across multiple providers instead of one only.
There may be something at the convergence or combination of some of the existing approaches or new tech that might be ready for the mainstream..
- Building momentum by starting as a MVNO in a segment of the market where you can secure and build demand and grow services. There are some interesting data only services out there.
- Perhaps a LoRa play. Great range. There seems to be some interesting gear that will pair Lora via Bluetooth to any phone. Software might be needed. Maybe there could be Lora beacons to assist with coverage.
- What’s possible with other open spectrums? Are there other unused spectrums that could be used in interesting ways, for exa
- Begin with cheaper 3G and now 4g networks only like new providers.
- Wifi calling overlap / voip coverage.
- Could there be a way to setup a peer routing network between devices and to their internet connections?
- Crowdsource/coop/non-profit focused on revenue and sustainability but not paying profit to shareholders. Non profit corporations do not mean mean people can’t be paid extremely well for their work. Maybe there is a meaningful social entrepreneurship angle here. Might be some funding available too to provide access to those who are priced out of the market.
- Maybe there is a way to use the enhanced networking capabilities of Linux only phones, or alternative setups.
Maybe there’s a fintech type disruption on the horizon.
This would be a fun project to hack on with 5-10 skilled folks with complementary skills to cover the industry, legal, financial, operational and the hardware and software angles to see what might be possible today. There have to be folks with a lot of experience in Canada that might be worth seeking out before beginning that feel exactly like you do and I would say I do too.
How much cash investment would you start with?
I’d be curious to how easy it can be for a non technical consumer today to access an mvno service that is turnkey and sells 3-4gb a month tablet type data plan (currently $10-20/mo) that can handle calls and sms via a carrier grade voip for $20-30/mo might be an interesting. It’s not new, but it’s not always easy to setup unless you’re technical. Calls are secondary to data (WhatsApp calls) these days for many. Keeping it independent of one carrier doesn’t preclude the fact that existing carriers have likely dealt with this type of approaches before. But it’s ok to start and remain small and let growth be organic snarfed across multiple providers instead of one only.
There may be something at the convergence or combination of some of the existing approaches or new tech that might be ready for the mainstream..
- Building momentum by starting as a MVNO in a segment of the market where you can secure and build demand and grow services. There are some interesting data only services out there.
- Perhaps a LoRa play. Great range. There seems to be some interesting gear that will pair Lora via Bluetooth to any phone. Software might be needed. Maybe there could be Lora beacons to assist with coverage.
- What’s possible with other open spectrums? Are there other unused spectrums that could be used in interesting ways, for exa
- Begin with cheaper 3G and now 4g networks only like new providers.
- Wifi calling overlap / voip coverage.
- Could there be a way to setup a peer routing network between devices and to their internet connections?
- Crowdsource/coop/non-profit focused on revenue and sustainability but not paying profit to shareholders. Non profit corporations do not mean mean people can’t be paid extremely well for their work. Maybe there is a meaningful social entrepreneurship angle here. Might be some funding available too to provide access to those who are priced out of the market.
- Maybe there is a way to use the enhanced networking capabilities of Linux only phones, or alternative setups.
> Perhaps a LoRa play
Not even worth trying at all. LoRa link speeds will barely satisfy even the most frugal internet users, especially in a world where modern webpages can run into the megabytes.
Not even worth trying at all. LoRa link speeds will barely satisfy even the most frugal internet users, especially in a world where modern webpages can run into the megabytes.
Thanks, I haven't worked with it. Messaging alone might be useful for some folks.
Start burying fiber and erecting towers.
When it comes to mobile, I have some experience there as I wanted to do the same and looked into it at a high level. The main takeaway is that it appears to be impossible and nobody who's in power to make it possible has incentives to make it so.
The main problem with displacing incumbent telcos is that you need to run your own physical infrastructure - you can't really do so as an MVNO - no company will work against themselves and they will use various tactics (whether contractual, pricing-based or use technical limitations) to make sure you're not able to be "too" competitive with them.
Therefore you need your own infrastructure - we're talking leasing land, building towers and purchasing expensive radio equipment every few miles, and you need to cover the whole country before you launch because nobody's gonna subscribe to phone service that only works in a small, specific area, and even though technically using roaming to make up for coverage gaps is possible, see the previous paragraph for why it won't happen (at least not sustainably).
Then you've got the issue of spectrum. You can have all the land and towers and equipment in the world but in most countries you need to have a license to be able to broadcast on the regulated mobile spectrum. Those licenses are auctioned every so often, and the incumbent telcos are willing to throw their entire war chests at them because it's a guaranteed return over time.
We're not talking millions or hundreds of millions here (the latter is just what you'd need for spectrum), we're talking billions. This is beyond what VC will fund, and isn't a good match for VC anyway because it will take decades to recoup that investment, during which the regulatory or technological situation might change - a huge risk investors are unlikely to take without the guaranteed of fast returns (which are difficult to do if your objective is to fix what's wrong with the existing telcos rather than become just a different kind of rent-seeker).
But let's say you've got the spectrum and land? Fine, let's talk about equipment. You've basically got a handful of vendors such as Ericsson or Huawei to pick from, and they're all terrible in their own ways. The equipment extremely expensive and may not even be sold, instead merely rented to you at a large price tag. It's often full of security holes and a pain to work with - see http://2014.hackitoergosum.org/slides/day1_Hacking-telco-equ... for an example. It's also a patent minefield so building your own is extremely difficult, though if you're in a location where software patents aren't recognized, you could potentially avoid the issue by using software-defined radio "breaching" the patents in software, but you'll still have to keep a good legal team on retainer and a big "rainy day fund" because the equipment manufacturers won't go down without a fight - and that doesn't mention the actual capital and talent needed to replicate decades of R&D.
Finally, even if you've made it all the way through this and are now on a level to compete with an incumbent, they can out-compete you by selling the service at a loss, subsidized by nastiness elsewhere, whether unethical practices such as advertising or selling personal data, to outright fraud such as extra fees/charging for services not rendered/etc, knowing full well the law won't be enforced against them. This is why telcos are universally hated, but if the market prices have now aligned with the fact that it's subsidized by nasty behavior, there might be no way for an ethical player to compete.
The main problem with displacing incumbent telcos is that you need to run your own physical infrastructure - you can't really do so as an MVNO - no company will work against themselves and they will use various tactics (whether contractual, pricing-based or use technical limitations) to make sure you're not able to be "too" competitive with them.
Therefore you need your own infrastructure - we're talking leasing land, building towers and purchasing expensive radio equipment every few miles, and you need to cover the whole country before you launch because nobody's gonna subscribe to phone service that only works in a small, specific area, and even though technically using roaming to make up for coverage gaps is possible, see the previous paragraph for why it won't happen (at least not sustainably).
Then you've got the issue of spectrum. You can have all the land and towers and equipment in the world but in most countries you need to have a license to be able to broadcast on the regulated mobile spectrum. Those licenses are auctioned every so often, and the incumbent telcos are willing to throw their entire war chests at them because it's a guaranteed return over time.
We're not talking millions or hundreds of millions here (the latter is just what you'd need for spectrum), we're talking billions. This is beyond what VC will fund, and isn't a good match for VC anyway because it will take decades to recoup that investment, during which the regulatory or technological situation might change - a huge risk investors are unlikely to take without the guaranteed of fast returns (which are difficult to do if your objective is to fix what's wrong with the existing telcos rather than become just a different kind of rent-seeker).
But let's say you've got the spectrum and land? Fine, let's talk about equipment. You've basically got a handful of vendors such as Ericsson or Huawei to pick from, and they're all terrible in their own ways. The equipment extremely expensive and may not even be sold, instead merely rented to you at a large price tag. It's often full of security holes and a pain to work with - see http://2014.hackitoergosum.org/slides/day1_Hacking-telco-equ... for an example. It's also a patent minefield so building your own is extremely difficult, though if you're in a location where software patents aren't recognized, you could potentially avoid the issue by using software-defined radio "breaching" the patents in software, but you'll still have to keep a good legal team on retainer and a big "rainy day fund" because the equipment manufacturers won't go down without a fight - and that doesn't mention the actual capital and talent needed to replicate decades of R&D.
Finally, even if you've made it all the way through this and are now on a level to compete with an incumbent, they can out-compete you by selling the service at a loss, subsidized by nastiness elsewhere, whether unethical practices such as advertising or selling personal data, to outright fraud such as extra fees/charging for services not rendered/etc, knowing full well the law won't be enforced against them. This is why telcos are universally hated, but if the market prices have now aligned with the fact that it's subsidized by nasty behavior, there might be no way for an ethical player to compete.
The telcos here are terrible. I'd love to hear how HN users would think of bootstrapping a telco?
We've had a few startups try and disrupt things in Canada (e.g., Wind Mobile). They were funded by large global investment groups and eventually simply got acquired by the incumbent telcos.
The other approach I've seen is piggybacking on existing telco infrastructure. This is required by law since the Canadian government subsidizes telco infrastructure... It's not uncommon for the telcos to renege on their contractual/legal obligations. Plus, this doesn't create any redundant networks.
So: how else can one launch a telco? What creative solutions have folks seen?
I'm thinking: VOIP-only service that has direct access to the Internet backbone in Toronto, Vancouver, etc. so you can at least have completely separate service in major cities... Or crowdfunding 5G towers (though I imagine spectrum licensing is an issue here).
Would love to hear what other HN readers think.