Viral infection starts with a virus particle landing on a cell surface followed by penetration of the plasma membrane. Due to the difficulty of measuring the rapid motion of small-sized virus particles on the membrane, little is known about how a virus particle reaches an endocytic site after landing at a random location. Here, we use coherent brightfield (COBRI) microscopy to investigate early stage viral infection with ultrahigh spatiotemporal resolution. By detecting intrinsic scattered light via imaging-based interferometry, COBRI microscopy allows us to track the motion of a single vaccinia virus particle with nanometer spatial precision (<3 nm) in 3D and microsecond temporal resolution (up to 100,000 frames per second). We explore the possibility of differentiating the virus signal from cell background based on their distinct spatial and temporal behaviors via digital image processing. Through image postprocessing, relatively stationary background scattering of cellular structures is effectively removed, generating a background-free image of the diffusive virus particle for precise localization. Using our method, we unveil single virus particles exploring cell plasma membranes after attachment. We found that immediately after attaching to the membrane (within a second), the virus particle is locally confined within hundreds of nanometers where the virus particle diffuses laterally with a very high diffusion coefficient (∼1 μm2/s) at microsecond time scales. Ultrahigh-speed scattering-based optical imaging may provide opportunities for resolving rapid virus−receptor interactions with nanometer clarity.