This paper introduces a lightweight, wearable device for telemanipulation tasks. Combined with the previously developed Mechanical Hand Tracker (MHT), it enables tracking the position and orientation of each fingertip relative to the torso without relying on grounded systems like calibrated cameras, allowing the user to move freely. The device overcomes common issues associated with data gloves and computer vision methods. We propose a design methodology for a linkage mechanism that maintains full mobility of the upper limbs and fingers while adapting to various body sizes. Teleoperation tests in a pick-and-place scenario were conducted to demonstrate its functionality.