With the rapid expansion of Internet of Thing (IoT) dependency, the necessity of lightweight communication is also increasing due to its constrained capabilities. Focusing on the limited capabilities of the devices, this research presents the design of a novel lightweight weight protocol called Enhanced Lightweight Security Gateway Protocol (ELSGP) based on a developed distributed computation model of the IoT layer. This distributed computational model introduces a new type of node called a Sub-server to assist edge layer servers and IoT devices in computation and act as a primary gateway of dependent IoT nodes. This thesis then introduces six features of ELSGP with developed algorithms that include access token distribution and validation, authentication and dynamic interoperability, attribute-based access control, traffic filtering, secure tunneling, and dynamic load distribution and balancing. Considering the variability of system requirements, ELSGP also outlines how to adopt a system-defined policy framework. For fault resiliency, this thesis also presents fault mitigation mechanisms, especially Trust and Priority Impact Relation for Byzantine, Cascading, and Transient faults. To validate the performances, we have simulated the protocol with an example scenario and extracted results for the several performance metrics. Based on the findings from the performance evaluation, further analysis of the protocol and future research directions are outlined. Undoubtedly, ELSGP could be a feasible solution for fulfilling the IoT-dependent system requirements.