Posted on

Q-Routing Revisited (Part 2)

In Part 1 we have recalled how Bellman-Ford, a distance vector algorithm, works to find shortest paths in a directed, weighted network. Now we want to understand in which aspects Q-Routing[1]Boyan, J. and Littman, M. (1994). Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach. modifies Bellman-Ford.

Difference 1: path relaxation steps performed asynchronously and online?

Difference 2: metric describing the path “quality”.

References

References
↑1 Boyan, J. and Littman, M. (1994). Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach.
Posted on

Learning of Routing Revisited

Before pairing the terms Deep Learning and Optimization of Network Traffic it is worthwhile to revisit what we know about learning in the context of routing traffic – or: what we know about routing algorithms and their inherent optimization component. The latter already indicates that finding an optimal route involves techniques that seek to incrementally improve some value until it is found (or does not change anymore). To this extent it appears that the problem of finding good routes had involved a form of learning[1]Learning in the sense of looking at, comparing and ranking data structures being presented and identifying an optimal solution (optimization, finding a recipe). However, not so much in the sense of … Continue reading.

We want to revisit the main concepts here. We will also revisit proposals such as Q-Routing[2]Boyan, J. and Littman, M. (1994). Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach.. And we want to round it up with recent proposals such as the MIND architecture[3]Geng, Y. (2017). MIND: Machine Learning based Network Dynamics. Talk at the Open Networking Summit 2016.. We also want to use theory from the dynamic control systems community on network architecture tradeoffs[4]Matni, N., Tang, A. and Doyle, J. (2015). A Case Study in Network Architecture Tradeoffs. ACM Sigcomm Symposium on SDN Research (SOSR) 2015. to see what the theory can do for us when different kinds of learning are involved. Please stay tuned.

References

References
↑1 Learning in the sense of looking at, comparing and ranking data structures being presented and identifying an optimal solution (optimization, finding a recipe). However, not so much in the sense of taking in new data (experience, training on a feedback signal from the environment, a reflection about how one has done) which is used to alter the function in order to do better. The latter requires changes to oneself (learning!) which has to take place in time. The former is kind of fixed (static) and in a narrow sense is stripped off the possibility of taking in feedback for adaptation after the optimum has been found, thus is not learning – i.e. transforming – in a strict sense which we will apply from here on now.
↑2 Boyan, J. and Littman, M. (1994). Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach.
↑3 Geng, Y. (2017). MIND: Machine Learning based Network Dynamics. Talk at the Open Networking Summit 2016.
↑4 Matni, N., Tang, A. and Doyle, J. (2015). A Case Study in Network Architecture Tradeoffs. ACM Sigcomm Symposium on SDN Research (SOSR) 2015.
Posted on

Deep Learning: Applications in Telecommunication Companies

Deep Learning[1]For example this Review: LeCun, Y., Bengio, Y. and Hinton, G. (2015). Deep learning. Nature. doi:10.1038/nature14539. is en vogue across all businesses. Also Telecommunications companies have (re-)shifted their attention to this topic. While the recent performance increases based on Deep Learning techniques have to be seen in the context of the tasks for which the techniques have been developed for[2]From the review in Nature referenced above: “Deep convolutional nets have brought about breakthroughs in processing images, video, speech and audio, whereas recurrent nets have shone light on … Continue reading, the question of how these tasks / challenges relate to the telecommunications business is worthwhile.

This is a list of telco task domains[3]Acker, O., Blockus, A. and Pötscher, F. (2013). Benefiting from data. pwc Strategy& Report. that could benefit from the strengths of machines, software and statistical inference:

  • Optimizing routing and quality of service by analyzing network traffic in real time
  • Analyzing call data records in real time to identify fraudulent behavior immediately
  • Allowing call center reps to flexibly and profitably modify subscriber calling plans immediately
  • Tailoring marketing campaigns to individual customers using location-based and social networking technologies
  • Using insights into customer behavior and usage to develop new products and services

This list can be extended and will be. What is of interest here is to what extent Deep Learning approaches can be successfully applied to these domains. As each of these domains has its own characteristics we shall focus on them individually – starting with the networks, routing, traffic control, etc. domain.

References

References
↑1 For example this Review: LeCun, Y., Bengio, Y. and Hinton, G. (2015). Deep learning. Nature. doi:10.1038/nature14539.
↑2 From the review in Nature referenced above: “Deep convolutional nets have brought about breakthroughs in processing images, video, speech and audio, whereas recurrent nets have shone light on sequential data such as text and speech.”
↑3 Acker, O., Blockus, A. and Pötscher, F. (2013). Benefiting from data. pwc Strategy& Report.