Due to new emerging applications and technologies, data traffic has increased dramatically. However, wireless network spectrum resources are very limited. To overcome this challenge, optimal network control, which can regulate optimal transmission times for network clients and support the largest set of client traffic rates while maintaining network stability, is required. However, the most optimal network control policies have been developed under light-tailed (LT) traffic conditions. When arrival traffic involves heavy-tailed (HT) traffic, the network fails to achieve queue stability under these policies. Thus, for a single-hop network, distributed maximum weight (Max-Weight) scheduling, which solves the above issue by adding alpha power on the queue backlog, has been proposed. Nevertheless, even though maximum weight-alpha scheduling overcomes this existing issue, it still must to decide the value for different kinds of arrival traffic, which is not easy to implement in a real environment. Hence, to further solve the problems faced here, we propose a time-average stochastic gradient scheduling algorithm (TASGSA), which decouples the queue-length up- date process and the dual-variable update process in such a way that both queue stabilities can be achieved. Similarly, for a multi-hop network, we propose a time-average stochastic gradient routing algorithm (TA-SGRA), which operates under similar principle but considers a cross-layer network system model. In addition, in order to explore the convergence and stability of the proposed algorithms, we use the ordinary differential equation (ODE) method since the HT traffic has an unbounded mean and variance. Finally, simulation results are presented to verify theoretical results for the proposed TA-SGSA and TA-SGRA algorithms, whereby an LT traffic and an LT routing ow queue can share the network with bounded mean and bounded variance in queue length, even in the presence of HT traffic and an HT routing flow queue.