Monday, December 24, 2012

TEST VOICE GATEWAYS T1/PRI

The number of issues that could arise in your VoIP deployment involving voice gateways are too numerous to capture in one blog.  The issues and variants are probably too numerous to capture in one blog series.  So, my intention with this series is to simply touch on methods I use for testing some (not all) issues I have come across with voice gateways.  Part 1 of this series will focus on how an admin can configure the dial plan in such a way that allows them to "seize" or force calls out any individual T1/E1 voice circuit in their Cisco Unified Communications Manager solution.

Background

 It could be that you don't know what I mean by "seizing" a channel or that you wonder why I keep putting "seize" in quotes.  On many PBX systems there is a way for the admin of the system to force an individual call out any voice circuit attached to the system.  It is a way for the admin to test physical circuits and validate call path.  Different admins and PBX vendors use different terms for this but "seizing" the trunk seems to be a common colloquialism.  I use quotes when referring to "seizing" a trunk with Cisco Unified Communication Manager (CUCM) simply because the concept doesn't inherently exist in the product.  If you want to implement a feature which allows admins to selectively send calls out a particular voice circuit, you will have to design and build it into your dial plan.

Example Dial Plan Scenario

While CUCM dial plan configurations vary from environment to environment, there are common elements across all designs:
  • Devices (phones) are configured with Calling Search Spaces (CSS)
  • Calling Search Spaces contain one or more partitions
  • Partitions contain one or more route patterns
  • Route patterns are configured (most of the time) to use route lists
  • Route lists contain one or more route groups
  • Route groups contain one or more gateways or trunks
Gateways basically bridge call legs between administratively separated domains (in the case of IP-IP gateway) or domains separated by signal methodology (i.e. MGCP and TDM).  The gateway or trunk destination is not an important consideration for the method discussed.
A sample dial plan scenario is provided in the following figure:
dpscenario1

There is nothing extraordinary in the example above, a user in Washington, D.C. dials the directory information line for the District of Columbia and assuming it isnt a blocked pattern the call will use a geographically designated route list. The route list may contain gateways in other geographic areas for redundancy purposes or not. In Chicago, a separate user tries to reach the same directory number may use a separate route list (assuming least cost routing is not implemented).

The Problem Scenario

Assume that the CUCM administrator is located in Chicago and wants to test call routing behavior using the gateways located in D.C. There are several solutions to this question, assuming that the dial plan design isnt that robust the answer most commonly used would be to reconfigure a test phone in Chicago with the D.C. calling search space and place the test call.
Now, assume that the Chicago administrator wants to test an outbound call through a D.C. PRI on the second gateway in Route Group 2 (which is the 4th gateway in the outbound selection list). In many cases this is more difficult and one may juggle route group priority orders to accomplish this.
Administrators may need to selectively test new PRIs added to the system, PRIs that may have voice quality issues, PRIs that may be rejecting long distance calls, or PRIs that have just undergone maintenance. Expanding on this care and feeding concept, an administrator may want to test individual analog circuits, or inter-PBX connections, analog termination on a video MCU, or any other critical system resource.
Using the standard approach the change needed to make a single test is overly cumbersome, may require coordination (i.e. Change Control window), and may introduce unexpected consequences.

A Possible Solution

The NetCraftsmen solution to this administrative burden is to build an overlay dial plan where specific patterns are used that allow a single phone to directly dial a remote party using any gateway configured in the Cisco IP telephony environment.
The basics:
  • Create trunk test route groups
  • Assign trunk test route groups to trunk test route lists
  • Assign one gateway to one trunk test group
  • Create pattern designations to uniquely identify trunk test codes
  • Place trunk test codes in a partition visible from all IP phone calling stations
  • Assign Forced Authorization Codes to provide a level of authorization
To illustrate, look at the following figure which illustrates the overlay concept using the previous example.

dpscenario2

Building the Route List - MGCP Environments

With CUCM 4.1 and later an administrator can assign a single gateway to more than one route group so this allows for two optional configurations when building a route list to work with the Trunk Access methodology.
First, all route groups can be configured in a 1:1 relationship with gateways. So, gateway1 will be part of Route Group1 and Route Group1 can be part of any Route List including the Trunk Testing Route List. The net benefit in the end game is fewer Route Groups which isnt usually a major issue in most environments.
The second option is to assign a gateway to normal call route groups in whatever way makes sense for the existing design and then assign the gateway to a separate Trunk Test route group specifically for use with trunk access testing. With the introduction of Standard Local Route Groups (SLRG) in CUCM 7.x, placing multiple gateways in a single route group is the option many folks will prefer.
In our example it doesn't matter which method an admin uses for their standard call path selection. We are going to create route groups with a single Gateway membership (note we assume two PRIs on each IOS gateway):
  • DC PSTN-localgw1 RG1: DC Gateway 1 is the only member
  • DC PSTN-localgw1 RG2: DC Gateway 2 is the only member
  • DC PSTN-localgw2 RG1: DC Gateway 3 is the only member
  • DC PSTN-localgw2 RG2: DC Gateway 4 is the only member
  • DC PSTN-localgw3 RG1: DC Gateway 5 is the only member
  • DC PSTN-localgw3 RG2: DC Gateway 6 is the only member
 Once assigned to route groups, these gateways can be placed in any production route list. In addition the same route groups can be assigned to route lists specifically used for trunk access testing:
  • DC Test-PRI01 RL: DC PSTN-localgw1 RG1 is the only member
  • DC Test-PRI02 RL: DC PSTN-localgw1 RG2 is the only member
  • DC Test-PRI03 RL: DC PSTN-localgw2 RG1 is the only member
  • DC Test-PRI04 RL: DC PSTN-localgw2 RG2 is the only member
  • DC Test-PRI05 RL: DC PSTN-localgw3 RG1 is the only member
  • DC Test-PRI06 RL: DC PSTN-localgw3 RG2 is the only member

 Building the Route List - H323/SIP Environments

With MGCP, each individual T1/E1 is registered to the CUCM cluster as a MGCP end-point. This is not the case with H.323 and SIP. You could have 16 T1-PRIs terminated on a H.323 gateway and the CUCM references the gateway by a single entry. In other words, the trunks connected to a H.323 gateway are transparent from a CUCM perspective.
This is not a problem, you just need to incorporate that known fact into your dial plan. You can do this on individual test patterns (discussed later) or you can accomplish this by configuring multiple route groups that contain the same H.323 gateway. Then, when you assign the route groups to a route list, modify the called party transformations to prefix digits that the remote H.323/SIP dial-peers will handle in a deterministic way.
For our purposes, we suggest creating a single route group for the H.323/SIP gateway that will be tested and then assign this route group as the only member of a route list used for testing. If you have multiple circuits on the H.323/SIP gateway then you'll use the test pattern to accommodate your dial-peer pattern matching needs. We'll get into this shortly.

Building the Dial Plan

All call routing starts with a digit pattern and the approach outlined in this blog is no different. As such it requires design consideration with respect to the existing dial plan solution. The first question that must be addressed is what partition should be used.
NetCraftsmen typically builds a partition that is dedicated to services which are only visible to IP phones on the CUCM cluster. For optimum flexibility it is recommended that the administrator use a partition that is visible from all IP phone calling search spaces, is not part of any ToD routing solution, and is not visible from any non-IP phone calling search space (i.e. gateway CSS or voicemail port CSS). For the purpose of continuing the example, the CL Svcs-Priv PT is used.
  • CL is used to designate scope, in this case cluster wide.
  • Svcs-Priv identifies this partition as a partition that provides special services that are only available to the local cluster.
After choosing the partition the next step is to identify a digit pattern which is scalable, flexible, and non-overlapping. The asterisk * is a useful digit in to avoid overlapping conditions (keeping in mind that using the asterisk in this way is not an original idea!).
Since scalability is critical and it is possible we may want to extend the usage of asterisk, a digit sequence could be added to designate a feature request. If two digits are used then the administrator can have 100 such features which should be sufficient in most cases. For this example we will reserve 0x for any telecommunication special service codes and assuming this is the first such application we will use *07.
Next a digit pattern is needed to uniquely assign to each gateway in the environment. Another consideration is possibly using a digit flag which identifies geographic location, cluster identification, or some other convention that makes sense to the organization or existing standard. Keep in mind that the future may bring changes that quickly introduce a numbering standard to obsolescence (e.g. using private IP address schemes that try to use octets as site designations).In this example we will use two digits which allows for 100 such configurations.
So, given our standard:
  • *0701 will use DC Test-PRI01 RL
  • *0702 will use DC Test-PRI02 RL
  • *0703 will use DC Test-PRI03 RL
  • *0704 will use DC Test-PRI04 RL
  • *0705 will use DC Test-PRI05 RL
  • *0706 will use DC Test-PRI06 RL
The patterns as they are listed above are not sufficient to satisfy the needs of the CUCM digit analysis subsystem. To make these patterns functional we will actually define a pre-dot digit truncation method, allow any number of digits following the pattern, and define a termination string. With these new configuration changes the pattern for DC Test-PRI01 RL is *0701.!#, the pattern for DC Test-PRI02 RL is *0702.!#, and so on.
The idea is that the administrator will supply all digits needed by the PSTN to route the call. So, if the administrator was going to call toll free information on DC PRI 01, the test pattern is:
*070118005551212#
So, the above would match the route pattern: *0701.!# and the route pattern is configured to strip pre-dot, the gateway receives 18005551212.
Some other examples:
Test Pattern Usage Example
*0703411# Sending information calls (411) out PRI number 3
*0705914105551212# Sending calls to directory information service line for Maryland to a PBX or Centrex Intercomm switch that requires an offnet code of 9.
*070410100333015551212# Use the Sprint long distance PIC code to dial information in California using PRI number 4.
*07012025551212# Dial information in Washington D.C. as a local 10-digit number using PRI number 1.

Going back to our problem scenario: The administrator in Chicago wants to test a call to the D.C. information line using the fourth PRI in the trunk group they would go off hook and dial *07042025551212#. Note that the administrator does not need to specify a leading digit of 1 for long distance since the call is not long distance from the DC trunk group.

Building the Dial Plan - H.323/SIP Considerations

The previous section assumes that you are using MGCP gateways. For the most part, the examples laid out in the previous section are applicable to call agent controlled (e.g. MGCP) and gateway controlled (e.g. H.323) protocols. However, there are a few other things to consider with call signaling scenarios that use gateway controlled protocols.
You can still use the route pattern conventions as defined. The only thing you need to work out is what digit patterns would you like to prefix to the pattern so that the remote H.323 gateway knows which trunk it should use? Basically, come up with a pattern prefix that the CUCM sends to the gateway which aligns with a POTS dial-peer pointing to the trunk you wish to test.
An example dial-peer configuration on one of the D.C. voice gateways in our example could be:
dial-peer voice 79000 pots
 description Trunk Testing circuit 1
 destination-pattern 1990T
 port 0/0/0:23
 !
 dial-peer voice 79001 pots
 description Trunk Testing circuit 2
 destination-pattern 1991T
 port 0/0/1:23
 !
If an admin wishes to test the T1-PRI connected to port 0/0/1 on the example voice gateway, they could use the digit string: *07027035551212#. This would match route pattern: *0702.!#. In this route pattern you would use the called party transformation section to strip the pre-dot and prefix 1991. When the gateway receives the call setup, it will strip the 1991 and send the remaining digits to the PSTN for processing.
Note that you will want to keep your gateway's Class of Restriction (COR) configuration in mind for SRST-enabled gateways. If you don't assign a COR to the test pattern dial-peers but use a corlist configuration for your attached ephones, then you will introduce an opening to toll-fraud scenarios. So, if you use COR on your H.323 gateway, then configure a special COR for these test patterns (e.g. corlist specialTelecomm) and make sure that the COR isn't a member of any SRST ephone corlist.

Authorization Codes

Our solution uses a dial-plan overlay that the operations team can use for testing. Our objective is to minimize unnecessary reconfiguration of phones (i.e. CSS) or route list/route group membership. At the same time, we don't want just any user being able to learn about the trunk access codes and use them to bypass any COR configurations you have in your production dial-plan. Is this likely? Who knows. It is a possible toll-fraud scenario and in my mind, that means it can't be ignored.
The solution is actually quite simple. For all route patterns configured for trunk testing use Forced Authorization Codes (FAC). Assign a permissions level like "250". Then create FAC codes for each of your operations staff or groups of staff members (whatever works best for you). Make sure the FAC permission level used is "higher" or "more restrictive" then what you may already have in place for local, long distance, interlata, international, etc..
If you use FAC and a user was clever enough to figure out the test pattern they would be blocked. Unless, of course, the user also figured out the FAC code. That would reinforce an operational recommendation to review CDRs periodically and to enable storing FAC codes in CDRs. However, this is definitely a topic for another day!

Cisco CUCM Blocking Calls by Calling Party Number (ID)

From time to time the Cisco Unified Communications Manager (CUCM) administrator receives a request to block inbound calls to an organization based on the calling party number (CPN). This is commonly known as "black listing". Historically, admins could use translation profiles on dial-peers to address this need. With CUCM 8.0 the options available to an administrator have been expanded by leveraging a new route directive on translation patterns.
Background
The approach described in this blog is focused on filtering calls based on CPN as presented from the PSTN (via a voice gateway or CUBE). It does not address "black listing" for Cisco Mobility or other station-to-station or stations-to-PSTN use cases.
A typical use case is to block marketing calls, recruitment calls, or any other call that is generally unwanted. The idea is that the administrator creates a list of CPNs that should be blocked and anything else is permitted. Truly a basic concept.
Prior to CUCM 8.0, the most common method for black listing unknown inbound calls based on CPN was to leverage translation-profiles on a Cisco IOS gateway. With the release of CUCM 8.0, there is another option made available to administrators that I believe improves overall operational sustainability.
The Translation-Profile Method
The following diagram illustrates the basics of this method.

In the above example, a call is presented to the voice gateway by the PSTN. The gateway leverages a translation-profile on an ingress POTS dial-peer to "test" the CPN against a list of "blocked" numbers. If the CPN is in the block list then the call is rejected using an administratively defined message. If the CPN is not blocked, then the voice gateway will continue processing the call, matching the VoIP leg dial-peer, and all is well in the world.
An example configuration supporting the described call flow could be:
!Voice Translation Rule set
voice translation-rule 69
  rule 1 reject /^800.*/
  rule 2 reject /2025551000/
  rule 3 reject /7035551000/
!
!Voice Translation Profile Assignment
voice translation-profile all-blacklist
  translate calling 69
!
! Sample POTS Dial-Peer
dial-peer voice 39001 pots
 description Ingress From PSTN
 incoming called-number 301555....
 call-block translation-profile incoming all-blacklist
 call-block disconnect-cause incoming call-reject
 direct-inward-dial
 port 0/0/1:23
 forward-digits 10
!
The POTS dial-peer 39001 is handling ingress calls from the PSTN and it is configured to check the ISDN call setup information elements (IE) against a translation-profile to determine if the call is permitted. The translation profile "all-blacklist" identifies the translation-rule and the IE that should be evaluated. Specifically, the calling IE.
Translation rule "69" defines several patterns that should be rejected. Namely, any number that begins 800, calls from 2025551000, and calls from 7035551000.
The method just described is only applicable to SIP and H.323 configurations. It does not work with MGCP. An additional point of interest is that there is a limit of 15 rules in a translation-rule set. Fortunately, I have not ran up against this limit but I suspect that some people have.
CUCM Route Next Hop by Calling Party Number
In CUCM version 8.0, Cisco added the Hotline Feature. One configuration element added to support this feature is a new paramter on Translation Patterns. This new parameter may be used to instruct the CUCM digit analysis routine to evaluate the call by CPN rather than called party number (DNIS). This configuration parameter is called "Route Next Hop by Calling Party Number" and it can be used to facilitate "black listing" CPNs without requiring the administrator to use the Hotline Feature.
The following diagram illustrates a sample call flow where CPN filtering is facilitated by this new capability.

The call flow can be described as follows:
  1. The PSTN carrier presents the call to the voice gateway.
  2. The voice gateway processes the call and then relays call setup information to the CUCM.
  3. The gateway object in CUCM is configured to use CL_PSTN-In_CSS, which is used for the initial digit analysis step.
  4. A translation pattern in CL_PSTN-In_PT route partition is configured to capture any CPN by using the "!" digit pattern
    • Translation Pattern: !
    • Partition: CL_PSTN-In_PT
    • Calling Search Space: CL_PSTN-Screen_CSS
    • Route Option: Route this pattern
    • Route Option: Urgent Priority
    • Route Option: Route Next Hop by Calling Party Number
  5. Assuming that a CPN is present, CUCM will continue digit analysis using CL_PSTN-Screen_CSS.
  6. The CL_PSTN-Screen_CSS contains one partition, CL_PSTN-Screen_PT and this partition will contain allow and deny patterns. This is where the magic happens.
Before going into how we want to leverage CL_PSTN-Screen_PT, I want to point out something about what is happening at Step 4. I tried this configuration a few times in my lab without success and I realized two things. First, I need to work on my reading comprehension skills. Second, the term "Route Next Hop by Calling Party Number" should be read literally.
What is happening here is that the "!" pattern in CL_PSTN-In_PT is telling the CUCM "evaluate the CPN against the patterns in CL_PSTN-Screen_CSS". Originally, I was trying to add my blocking patterns to CL_PSTN-In_PT with the "Route Next Hop" flag and the "Block this pattern" flag set on the same pattern. This does not work.
Instead, what you need to do is create translation patterns in CL_PSTN-Screen_PT that are configured as normal translation patterns (i.e. do not check the "Route Next Hop by Calling Party Number" option). What the CUCM digit analysis process is going to do is take the CPN and compare it with the translation pattern(s) in CL_PSTN-Screen_PT.
Continuing our example, as you can see in the figure, we have defined the following translation patterns in CL_PSTN-Screen_PT:
  • "!": This pattern is essentially our explicit "allow all" pattern and the Route Option flag is set to "Route this Pattern". You need this to allow call setup to continue for patterns you want to allow through.
  • The following patterns are  configured with the Route Option flag "Block this pattern". So, calls from a CPN that matches any of these patterns is blocked:
    • "2025551000": Specific pattern match.
    • "800!": Any CPN that starts with 1800 or 800 is matched.
When blocking a pattern, the administrator can select one of several disconnect cause codes to send back to the PSTN carrier (via the voice gateway). Selecting "Call Rejected" may be the preferred option because is the system originating the call is automated (i.e. a marketing company's predictive dialer) then there is a possibility the system will act on the rejected inform message and remove the DNIS from their dialing target table.
Summary of Configuration Steps

In my lab, I used the following configuration procedures.
  1. Create partitions:
    • CL_PSTN-In_PT: This partition will hold the translations which flag "Route Next Hop" behavior.
    • CL_PSTN-Screen_PT: This partition will hold translations used to evaluate CPN.
  2. Create Calling Search Spaces:
    • CL_PSTN-In_CSS: This is the CSS assigned to the voice gateway and it contains the CL_PSTN-In_PT.
    • CL_PSTN-Screen_CSS: This is the CSS assigned to the patterns in CL_PSTN-In_PT. This CSS contains the CL_PSTN-Screen_PT.
    • CL_Tenant-Control_CSS: This CSS is assigned to patterns in CL_PSTN-Screen_PT and it contains partitions that have phone DNs and other patterns that route to devices or applications on the CUCM cluster.
  3. Assign the CL_PSTN-In_CSS to the voice gateway.
  4. Add the "Route Next Hop" translation "!" to CL_PSTN-In_PT. Select the CL_PSTN-Screen_CSS, enable call routing, and enable "Route Next Hop by Calling Party Number". NOTE: You can also apply calling or called party transformations at this step, as needed.

  5. Add the "allow all" translation "!" to CL_PSTN-Screen_CSS, enable call routing, and DO NOT enable "Route Next Hop by Calling Party Number".
  6. Add any "black list" translation. Enable call blocking, select disconnect cause reason, and you are good to go.
Considerations
I am not sure why Cisco opted to use a "next instruction" approach here. I am sure there is a good reason and it is probably related to how the Hotline feature is designed to operate. I did try to mix translations that were routing on called party and translations that had the "route next hop" option enabled. Thus, removing one of the prescribed steps. This doesn't work as one would hope. In my tests, the CUCM digit analysis always preferred routing by called party information if you mix patterns in the same CSS.
The configuration examples provided assume that the calls coming into the CUCM are presenting a valid CPN. If calls coming into the system are flagged as private then CPN is not provided. The pattern "!" won't help you and you may need to look at using a null (blank) pattern in CL_PSTN-In_PT and CL_PSTN-Screen_PT. (I have not tested this.)
Conclusions
I am still working through running the CUCM approach described in this blog through some of our production design scenarios for validation purposes. Thus far, the approach seems viable for those who use MGCP gateways, would like to centralize call routing on the CUCM, or have ran into limits with the number of rules that can be assigned to a translation-rule set in IOS.

Cisco RTMT on Mac OS

CISCO RTMT on Mac OS X

  @ciscomonkey's article that is aptly titled "Real Time Monitoring Tool on Mac OS X" (http://www.ciscomonkey.net/rtmt-on-mac/)