title
APPROVE A CONTRACT WITH AT&T TO PROVIDE SWITCHED ETHERNET SERVICE AND NETWORK-ON-DEMAND FIBER SERVICES TO REMOTE SITES
recommended action
RECOMMENDATION
It is recommended that the City Council approve a motion to:
1. Authorize the purchase of the AT&T Switched Ethernet Service with Network on Demand (ASEoD) telecommunications service plan and support services in the amount of $433,850.40 (Attachment A) and authorize an additional contingency of $100,000 for a total amount of $533,850.40, for 60 months estimated to begin July 1, 2025, through June 30, 2030;
2. Approve findings which support and justify an exception to the competitive bid process; and
3. Authorize the City Manager to take necessary and appropriate actions to carry out the purpose and the intent of this motion.
body
Summary
The City owns and operates multiple remote sites utilized by various departments, including Fire, Police, Municipal Utilities, Economic Development, and Community Services. These sites are connected to the City’s core network via copper or fiber connections. The slow speeds associated with copper-based connections is insufficient for modern operational needs. This has particularly impacted fire stations, where network limitations, copper vandalism, and declining vendor support have created significant operational challenges for critical services, including 911 Emergency Alerts and the G2 Alerting System.
The majority of citywide telecommunication services are purchased using the State of California’s CALNET contract. AT&T is offering to provide fiber-based connection services at a substantially discounted rate that other CALNET vendors do not offer, with overall savings off their CALNET pricing of approximately $265,000. Implementing AT&T's Switched Ethernet on Demand service will deliver significant improvements, including speeds up to 1 Gbps, enhanced reliability, reduced latency, lower maintenance costs, and scalability across all remote sites citywide.
Staff recommend that the Council approve the proposed service agreement with AT&T for its fiber-connection service plan, Switched Ethernet Service with Network on Demand, in the amount of $433,850.40, for a term of 60 months, estimated to begin July 1, 2025, through June 30, 2030. Staff further recommend that the Council authorize an additional contingency of $100,000, for a total expenditure authority of $533,850.40.
DISCUSSION
Background
The City owns and operates several remote sites that are connected to the City’s core network using various mediums, primarily copper or fiber-based connections. These sites are utilized by various departments, including Fire, Police, Municipal Utilities, Economic Development, and Community Services. A significant number of locations are hindered by their reliance on copper-based connections, which provide inadequate speeds equivalent to a dial-up phone connection. Copper-based connections are no longer adequate, especially at first responder facilities, due to low bandwidth which fall short of meeting first responder needs to support modern operational and service requirements in recent years.
The City’s fire stations have been severely hindered by a lack of sufficient network bandwidth, a rise in copper vandalism, and an overall decrease in vendor technician support services for copper-based connections. The Emergency Operations Center and remote fire stations throughout the City still rely on copper connections for 911 Emergency Alerts and the G2 Alerting System used to dispatch first responders at each firehouse. In lieu of a complete connection, a combination of AT&T copper T-1 (1.5 Mbps) and Comcast business-class coaxial internet (50/20 Mbps) connections are used. The Comcast connections have upload limitations, which have posed a problem with deploying or pushing out system and security updates to computers. The Comcast connection also requires a VPN connection to access core network resources, which slows down the network connection. Furthermore, copper T-1 lines have become prone to theft and vandalism, bottleneck network utilization, and can no longer support the City’s network bandwidth needs.
In response to these issues, Fire and Information Technology (IT) departments initiated a major replacement of network infrastructure under the National IPA Cooperative Agreement for General Information Technology Solutions and Services with CDW Government, approved by Motion 2020-03-10-1401, migrating equipment from an outdated copper cabling system to a modern fiber system. Beginning in 2024, the initial technology-readiness phase was to rewire network cabling for sites and procure upgraded hardware, including VoIP phones and wireless access points, compatible with a modern fiber system. Meanwhile, fire stations remain on the old T-1 copper system while project phases for network cabling and network hardware equipment replacements are underway and nearing completion.
The final phase and basis of this request is to enter into a new agreement with AT&T for the Switched Ethernet on Demand service plan, which will be available to all remote sites with fiber-based connections. This upgraded AT&T service plan will provide increased speeds (with port connections up to 1 Gbps as needed), improve reliability and reduce latency, lower service and maintenance costs, and provide scalability to fire stations and all AT&T-supported remote sites citywide.
This request before the Council authorizes a new 60-month contract with AT&T for ASEoD services, estimated to begin July 1, 2025, and end June 30, 2030.
Present Situation
The AT&T ASEoD contract is a fiber-connection service plan that can consolidate and eliminate the use of the AT&T copper T-1 and Comcast coax connections and allow the City to scale bandwidth and services to technology-ready remote locations, as needed. With a fiber connection, speeds are synchronous, meaning that the download and upload speeds are the same, thereby reducing latency and delays in data transfer and providing a more consistent performance. The ASEoD service plan offers scalable, synchronous fiber connections with baseline speeds of 100 Mbps (upgradable to 1 Gbps)as needed) and enables a direct connection to the City’s core network eliminating the need for a VPN, significantly improving performance of network connections to remote sites that depend on copper T-1 lines. This also benefits and enables the IT Department to push out applications and changes to the system much faster than processes allow today. Furthermore, the service is scalable with same-day response from an AT&T support team for any repairs, upgrades, or increases in bandwidth/connection speeds when needed, and will be available to all AT&T-supported remote sites.
New fiber connections and transfer service plans are initially planned for all fire stations, the Emergency Operation Center, Stockton Animal Shelter, Civic Center, two remote sites for Municipal Utilities (Stockton Water Conservation and Delta Water Supply), and other remote sites managed by the IT Department. A total of 21 locations are included in the proposed ASEoD service plan. Staff also recommend contingency for newly acquired buildings and/or future transition of additional remote locations not initially included in the proposed plan, for an estimated amount of $100,000.
Without the ASEoD service plan, fire stations will continue to experience down-time and service disruptions due to T-1 copper line issues of theft, vandalism, weather damage, and limited vendor support available for specialized copper line repairs. There are no after-hours copper line repair technicians available under current service plans. The proposed solution of moving remote sites to the ASEoD contract provides benefits to all AT&T supported remote locations with same-day repair services and cost savings over existing AT&T and State of California CALNET contract prices, and in the case of fire stations, there would be significant improvement in network performance from eliminating the use of copper-based connections. Other alternatives considered had slower data transfer speeds, and vendors indicated they did not offer rates lower than CALNET pricing.
Table 1 shows pricing by department for the proposed 60-month ASEoD service plan and recommended contingency. It also shows estimated cost savings over standard AT&T CALNET pricing, a contract which the City currently uses for most network telecommunication services.
Table 1: Proposed ASEoD Pricing and Estimated Cost Savings vs. CALNET Pricing
|
Department |
ASEoD Annual Cost |
ASEoD Total Cost (60 Months) |
Estimated Cost Savings (vs. CALNET) |
|
Fire |
$57,455.88 |
$287,279.40 |
$168,450.60 |
|
IT |
$17,253.48 |
$86,267.40 |
$65,862.60 |
|
MUD |
$8,040.48 |
$40,202.40 |
$20,517.60 |
|
Police |
$4,020.24 |
$20,101.20 |
$10,258.80 |
|
Contingency |
$20,000.00 |
$100,000.00 |
|
|
Total |
$106,770.08 |
$533,850.40 |
$265,089.60 |
Staff recommend that the Council approve a new service agreement with AT&T for a fiber-connection service plan, Switched Ethernet Service with Network on Demand, in the amount of $433,850.40, for a service term of 60 months, estimated to begin July 1, 2025, through June 30, 2030. A $100,000 contingency is included to support future needs such as additional buildings or sites for a total expenditure authority of $533,850.40.
Findings:
Stockton Municipal Code section 3.68.070(A)(4) provides an exception to the competitive bidding process for single available source vendors in cases where the City Council has approved findings that support and justify the exception. Below are proposed findings which, if adopted by Council, constitute sufficient grounds for making such an exception:
1. Telecommunications services require specialized expertise and resources that the City does not possess in-house, and require an experienced firm to provide such services. Furthermore, the City does not have specialized personnel or equipment to perform the work required to maintain the telecommunication system.
2. AT&T has been able to provide a deeper discount than what is available from CALNET services offered by AT&T as well as Comcast.
3. This procurement provides substantial savings for monthly services to be provided to multiple fire stations. AT&T offers on-demand bandwidth that allows us to throttle speeds as needed, while other service providers lock in speeds.
FINANCIAL SUMMARY
The services to be procured through AT&T are for the fiber telecommunication ASEoD service plan and support services for the City’s network fiber connection to remote sites. The ASEoD service plan is proposed in the amount of $433,850.40, with a service term of 60 months, to begin in FY 2025-26. Staff further request an additional contingency of $100,000.00, to support future needs such as newly acquired buildings and/or department requests for transition of additional remote locations for dedicated business services not included initially in the proposed plan, for a total expenditure authority of $533,850.40 (Table 2).
Services to remote sites initiated by this agreement will be funded through the Technology Internal Service Fund (ISF) for enterprise services or department accounts for dedicated business services and will not exceed the budget authority of future budgets approved through the annual budget process. Additionally, the Fire and Police departments will impact the General Fund, while the budgets for the IT and MUD departments have been approved in accordance with the annual budget process.
Table 2: Proposed Purchase Authority and Accounts
|
Department |
Fund |
Account |
ASEoD Total Cost (60 Months) |
|
Fire |
General Fund |
4020-000-640005-100-000-20-40-000-000- |
$287,279.40 |
|
IT |
Technology ISF |
3020-000-640005-510-511-00-30-000-000- |
$86,267.40 |
|
MUD |
Water Utility Fund |
6514-000-640005-600-000-00-65-000-000- (WFO); 6513-511-640005-600-000-00-65-000-000- (DWTP) |
$40,202.40 |
|
Police |
General Fund |
3519-320-640005-100-000-20-35-000-000- |
$20,101.20 |
|
Contingency |
Technology ISF or Department Funds* |
3020-000-640005-510-511-00-30-000-000- |
$100,000.00 |
|
Total |
|
|
$533,850.40 |
*Note: If authorized, contingency expenditures will not exceed amounts approved through the annual budget process and will be funded through the Technology ISF for citywide services or by department accounts for any dedicated business services, consistent with current ISF program operations and rate methodologies.
Attachment A - AT&T ASEoD Contract