English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

People with blindness and low vision (pBLV) encounter substantial challengeswhen it comes to comprehensive scene recognition and precise objectidentification in unfamiliar environments. Additionally, due to the visionloss, pBLV have difficulty in accessing and identifying potential trippinghazards on their own. In this paper, we present a pioneering approach thatleverages a large vision-language model to enhance visual perception for pBLV,offering detailed and comprehensive descriptions of the surroundingenvironments and providing warnings about the potential risks. Our methodbegins by leveraging a large image tagging model (i.e., Recognize Anything(RAM)) to identify all common objects present in the captured images. Therecognition results and user query are then integrated into a prompt, tailoredspecifically for pBLV using prompt engineering. By combining the prompt andinput image, a large vision-language model (i.e., InstructBLIP) generatesdetailed and comprehensive descriptions of the environment and identifiespotential risks in the environment by analyzing the environmental objects andscenes, relevant to the prompt. We evaluate our approach through experimentsconducted on both indoor and outdoor datasets. Our results demonstrate that ourmethod is able to recognize objects accurately and provide insightfuldescriptions and analysis of the environment for pBLV.

A Multi-Modal Foundation Model to Assist People with Blindness and Low Vision in Environmental Interaction