Sustained observations are required to determine the marine plastic debris mass balance and to support effective policy for planning remedial action. However, observations currently remain scarce at the global scale. A satellite remote sensing system could make a substantial contribution to tackling this problem. Here, we make initial steps towards the potential design of such a remote sensing system by: (1) identifying the properties of marine plastic debris amenable to remote sensing methods and (2) highlighting the oceanic processes relevant to scientific questions about marine plastic debris. Remote sensing approaches are reviewed and matched to the optical properties of marine plastic debris and the relevant spatio-temporal scales of observation to identify challenges and opportunities in the field. Finally, steps needed to develop marine plastic debris detection by remote sensing platforms are proposed in terms of fundamental science as well as linkages to ongoing planning for satellite systems with similar observation requirements.