지식 집약형 시각 질의응답(KI-VQA) 벤치마크는 제공된 이미지를 넘어서는 외부 정보를 요구하여 질문에 답함으로써, 비전-언어 모델(VLM)을 멀티모달 지식 어시스턴트로 평가한다. KI-VQA는 지시 표현(referring expression) 이해, 시각적 그라운딩(visual grounding), 객체 인식, 지식 검색, 추론 등 여러 하위 문제를 포함하지만, 기존 벤치마크는 일반적으로 최종 과제 정확도만 보고하여 실패가 어디에서 발생하는지를 가리는 경우가 많다. 전체 KI-VQA 파이프라인을 분석하기 위해, 우리는 1) 언어 기반 시각적 그라운딩, 2) 객체 식별, 3) 지식 검색 및 추론을 분리하는 단계별 데이터 어노테이션이 포함된 진단 벤치마크인 CRAG-MM-Diagnostics를 제안한다. 우리는 완전 파라미터형 및 검색-증강 VLM을 평가하고, 대상 ROI, 개체 이름, 시각적 복잡도 점수 등 새로 수집한 메타데이터를 활용하여 세밀한 분석을 제공한다. 우리의 결과는 지식 검색 및 추론이 주요 병목임을 시사하지만, VLM이 대상 객체 식별에 어려움을 겪거나 이미지 검색기가 텍스트 단서를 통합하는 데 어려움을 겪는 등 KI-VQA 파이프라인의 다른 부분에서도 문제가 있음을 강조한다. 이러한 발견은 현재 KI-VQA 시스템의 근본적 한계를 드러내고 단계 인식 평가(stage-aware evaluation)의 필요성을 촉진한다. 마지막으로, 우리는 이러한 발견을 활용하여 이미지 검색 전에 시각적 그라운딩 모듈을 통합해 대상을 크롭하는 그라운딩 기반 이미지-텍스트 RAG 파이프라인을 제안하며, GPT-5와 Qwen의 정확도를 각각 13.3 및 8.5 퍼센트포인트 향상시킨다.
비전-언어 모델(VLM)의 시각적 인코더와 언어 디코더가 서로 다른 컴퓨팅 노드에서 실행될 때, 중간 시각적 토큰 임베딩은 내부 활성화가 아닌 통신되는 페이로드가 됩니다. 우리는 기계가 소비하는 이러한 중간 텐서를 AI 트래픽(AI traffic)이라 부르며, 이를 표준화된 학습(training) 불필요 코덱으로 어디까지 압축할 수 있는지 질문을 던집니다. 우리는 가중치, 프롬프트, 생성을 변경하지 않은 채, 주요 시각적 토큰 표현과 DeepStack 특징 스트림으로 구성된 Qwen3-VL-8B-Instruct 비디오 질문 답변 파이프라인의 전체 시각적 인터페이스에 ISO/IEC 15938-17 신경망 코딩(NNC) 왕복 처리를 삽입하고, 넓은 전송률 범위에 걸쳐 양자화 매개변수(QP)를 탐색합니다. 폐쇄형 Video-MME 정확도는 전송되는 BF16 텐서의 98% 감소까지 압축되지 않은 기준선에 가깝게 유지되다가 그 이후에야 급격히 저하되며, 개방형 MLVU 생성 역시 LLM 판정관 평가에서 동일한 고원-급락(plateau-and-collapse) 양상을 보입니다. 이러한 견고성은 무손실에 가까운 재구성 때문이 아닙니다. 디코딩된 텐서는 심하게 이산화되어 있고, 상당한 행별 상대 L2 오차를 수반하며, 원본보다 눈에 띄게 더 가파른 특이값 감쇠를 보입니다. 따라서 다운스트림 추론은 정확한 부동소수점 값이 아닌 거친 구조와 상대적 기하학적 형태에 의존하며, 이는 AI 트래픽 코덱에서 율-왜곡(rate-distortion) 최적화보다 율-과업(rate-task) 최적화가 필요함을 뒷받침합니다.
대규모 비전-언어 모델(LVLM)은 자동화된 안과 진단 분야에서 막대한 잠재력을 보여줍니다. 그러나 도메인 특화 지식의 부재로 인해 실제 임상 배치가 심각하게 저해되고 있습니다. 본 연구에서는 신뢰할 수 있는 의료 추론을 방해하는 두 가지 구조적 결함을 규명합니다. 첫째는 범용 비주얼 인코더가 미세한 병리학적 단서(예: 미세동맥류)를 분해해 내지 못하는 '인식 격차(Perception Gap)'이며, 둘째는 트랜스포머(transformer)의 더 깊은 레이어에서 희소한 시각적 근거가 방대한 언어 사전 확률에 점진적으로 압도되어 근거 없는 환각(hallucination)으로 이어지는 '추론 격차(Reasoning Gap)'입니다. 이러한 격차를 해소하기 위해, 본 논문에서는 딥 엑스퍼트 인젝션(Deep Expert Injection) 메커니즘을 통해 망막 VLM을 전문가 지식에 앵커링하도록 설계된 데이터 효율적인 프레임워크인 EyExIn을 제안합니다. 제안하는 아키텍처는 시각적 표현을 해부학적 맥락을 위한 일반 스트림과 병리학적 의미를 위한 특화된 전문가 스트림으로 분리하는 전문가 인지 듀얼 스트림(Expert-Aware Dual-Stream) 인코딩 전략을 채택합니다. 높은 충실도의 통합을 보장하기 위해 무관한 배경 노이즈를 필터링하면서 미세한 병변 신호를 동적으로 증폭하는 시맨틱 적응형 게이트 융합(Semantic-Adaptive Gated Fusion) 모듈을 설계했습니다. 나아가, 융합된 시각적 특징을 잔차 편향(residual bias) 형태로 중간 LLM 레이어에 직접 통합함으로써 지속적인 '비전 앵커(Vision Anchors)'를 내장하는 적응형 딥 엑스퍼트 인젝션을 도입합니다. 이 메커니즘은 추론 스택이 시각적 근거에 엄격하게 그라운딩(grounding)된 상태를 유지하도록 강제하는 시각적 숏컷을 생성합니다. 4개의 벤치마크에 걸친 광범위한 실험을 통해 제안하는 모델(model)이 방대한 상용 독점 시스템을 일관되게 능가함을 입증했습니다. EyExIn은 도메인 특화 지식 임베딩(embedding)을 크게 향상시키고 안과 시각적 질의응답에서 최고 수준의 정밀도(precision)를 달성하여 신뢰할 수 있는 안과 AI의 발전을 촉진합니다.
Generating consecutive descriptions for videos, i.e., Video Captioning, requires taking full advantage of visual representation along with the generation process. Existing video captioning methods focus on making an exploration of spatial-temporal representations and their relationships to produce inferences. However, such methods only exploit the superficial association contained in the video itself without considering the intrinsic visual commonsense knowledge that existed in a video dataset, which may hinder their capabilities of knowledge cognitive to reason accurate descriptions. To address this problem, we propose a simple yet effective method, called Visual Commonsense-aware Representation Network (VCRN), for video captioning. Specifically, we construct a Video Dictionary, a plug-and-play component, obtained by clustering all video features from the total dataset into multiple clustered centers without additional annotation. Each center implicitly represents a visual commonsense concept in the video domain, which is utilized in our proposed Visual Concept Selection (VCS) to obtain a video-related concept feature. Next, a Conceptual Integration Generation (CIG) is proposed to enhance the caption generation. Extensive experiments on three publicly video captioning benchmarks: MSVD, MSR-VTT, and VATEX, demonstrate that our method reaches state-of-the-art performance, indicating the effectiveness of our method. In addition, our approach is integrated into the existing method of video question answering and improves this performance, further showing the generalization of our method. Source code has been released at https://github.com/zchoi/VCRN.
Multimodal Large Language Models (MLLMs) have emerged to tackle the challenges of Visual Question Answering (VQA), sparking a new research focus on conducting objective evaluations of these models. Existing evaluation methods face limitations due to the significant human workload required to design Q&A pairs for visual images, which inherently restricts the scale and scope of evaluations. Although automated MLLM-as-judge approaches attempt to reduce the human workload through automatic evaluations, they often introduce biases. To address these problems, we propose an Unsupervised Peer review MLLM Evaluation framework. It utilizes only image data, allowing models to automatically generate questions and conduct peer review assessments of answers from other models, effectively alleviating the reliance on human workload. Additionally, we introduce the vision-language scoring system to mitigate the bias issues, which focuses on three aspects: (i) response correctness; (ii) visual understanding and reasoning; and (iii) image-text correlation. Experimental results demonstrate that UPME achieves a Pearson correlation of 0.944 with human evaluations on the MMstar dataset and 0.814 on the ScienceQA dataset, indicating that our framework closely aligns with human-designed benchmarks and inherent human preferences.
Image quality scoring and interpreting are two fundamental components of Image Quality Assessment (IQA). The former quantifies image quality, while the latter enables descriptive question answering about image quality. Traditionally, these two tasks have been addressed independently. However, from the perspective of the Human Visual System (HVS) and the Perception-Decision Integration Model, they are inherently interconnected: interpreting serves as the foundation for scoring, while scoring provides an abstract summary of interpreting. Thus, unifying these capabilities within a single model is both intuitive and logically coherent. In this paper, we propose Q-SiT (Quality Scoring and Interpreting joint Teaching), a unified framework that enables large multimodal models (LMMs) to learn both image quality scoring and interpreting simultaneously. We achieve this by transforming conventional IQA datasets into learnable question-answering datasets and incorporating human-annotated quality interpreting data for training. Furthermore, we introduce an efficient scoring & interpreting balance strategy, which first determines the optimal data mix ratio on lightweight LMMs and then maps this ratio to primary LMMs for fine-tuning adjustment. This strategy not only mitigates task interference and enhances cross-task knowledge transfer but also significantly reduces computational costs compared to direct optimization on full-scale LMMs. With this joint learning framework and corresponding training strategy, we develop Q-SiT, the first model capable of simultaneously performing image quality scoring and interpreting tasks, along with its lightweight variant, Q-SiT-mini. Experimental results demonstrate that Q-SiT achieves strong performance in both tasks with superior generalization IQA abilities.Project page at https://github.com/Q-Future/Q-SiT.
Adversarial attacks on a convolutional neural network (CNN) -- injecting human-imperceptible perturbations into an input image -- could fool a high-performance CNN into making incorrect predictions. The success of adversarial attacks raises serious concerns about the robustness of CNNs, and prevents them from being used in safety-critical applications, such as medical diagnosis and autonomous driving. Our work introduces a visual analytics approach to understanding adversarial attacks by answering two questions: (1) which neurons are more vulnerable to attacks and (2) which image features do these vulnerable neurons capture during the prediction? For the first question, we introduce multiple perturbation-based measures to break down the attacking magnitude into individual CNN neurons and rank the neurons by their vulnerability levels. For the second, we identify image features (e.g., cat ears) that highly stimulate a user-selected neuron to augment and validate the neuron's responsibility. Furthermore, we support an interactive exploration of a large number of neurons by aiding with hierarchical clustering based on the neurons' roles in the prediction. To this end, a visual analytics system is designed to incorporate visual reasoning for interpreting adversarial attacks. We validate the effectiveness of our system through multiple case studies as well as feedback from domain experts.
Automatically determining whether a text and a corresponding image are semantically aligned is a significant challenge for vision-language models, with applications in generative text-to-image and image-to-text tasks. In this work, we study methods for automatic text-image alignment evaluation. We first introduce SeeTRUE: a comprehensive evaluation set, spanning multiple datasets from both text-to-image and image-to-text generation tasks, with human judgements for whether a given text-image pair is semantically aligned. We then describe two automatic methods to determine alignment: the first involving a pipeline based on question generation and visual question answering models, and the second employing an end-to-end classification approach by finetuning multimodal pretrained models. Both methods surpass prior approaches in various text-image alignment tasks, with significant improvements in challenging cases that involve complex composition or unnatural images. Finally, we demonstrate how our approaches can localize specific misalignments between an image and a given text, and how they can be used to automatically re-rank candidates in text-to-image generation.
Research on 3D Vision-Language Models (3D-VLMs) is gaining increasing attention, which is crucial for developing embodied AI within 3D scenes, such as visual navigation and embodied question answering. Due to the high density of visual features, especially in large 3D scenes, accurately locating task-relevant visual information is challenging. Existing works attempt to segment all objects and consider their features as scene representations. However, these task-agnostic object features include much redundant information and missing details for the task-relevant area. To tackle these problems, we propose LSceneLLM, an adaptive framework that automatically identifies task-relevant areas by leveraging LLM's visual preference for different tasks, followed by a plug-and-play scene magnifier module to capture fine-grained details in focused areas. Specifically, a dense token selector examines the attention map of LLM to identify visual preferences for the instruction input. It then magnifies fine-grained details of the focusing area. An adaptive self-attention module is leveraged to fuse the coarse-grained and selected fine-grained visual information. To comprehensively evaluate the large scene understanding ability of 3D-VLMs, we further introduce a cross-room understanding benchmark, XR-Scene, which contains a series of large scene understanding tasks including XR-QA, XR-EmbodiedPlanning, and XR-SceneCaption. Experiments show that our method surpasses existing methods on both large scene understanding and existing scene understanding benchmarks. Plunging our scene magnifier module into the existing 3D-VLMs also brings significant improvement.
Real-time detection and prediction of extreme weather protect human lives and infrastructure. Traditional methods rely on numerical threshold setting and manual interpretation of weather heatmaps with Geographic Information Systems (GIS), which can be slow and error-prone. Our research redefines Extreme Weather Events Detection (EWED) by framing it as a Visual Question Answering (VQA) problem, thereby introducing a more precise and automated solution. Leveraging Vision-Language Models (VLM) to simultaneously process visual and textual data, we offer an effective aid to enhance the analysis process of weather heatmaps. Our initial assessment of general-purpose VLMs (e.g., GPT-4-Vision) on EWED revealed poor performance, characterized by low accuracy and frequent hallucinations due to inadequate color differentiation and insufficient meteorological knowledge. To address these challenges, we introduce ClimateIQA, the first meteorological VQA dataset, which includes 8,760 wind gust heatmaps and 254,040 question-answer pairs covering four question types, both generated from the latest climate reanalysis data. We also propose Sparse Position and Outline Tracking (SPOT), an innovative technique that leverages OpenCV and K-Means clustering to capture and depict color contours in heatmaps, providing ClimateIQA with more accurate color spatial location information. Finally, we present Climate-Zoo, the first meteorological VLM collection, which adapts VLMs to meteorological applications using the ClimateIQA dataset. Experiment results demonstrate that models from Climate-Zoo substantially outperform state-of-the-art general VLMs, achieving an accuracy increase from 0% to over 90% in EWED verification. The datasets and models in this study are publicly available for future climate science research: https://github.com/AlexJJJChen/Climate-Zoo.
Understanding a scene by decoding the visual relationships depicted in an image has been a long studied problem. While the recent advances in deep learning and the usage of deep neural networks have achieved near human accuracy on many tasks, there still exists a pretty big gap between human and machine level performance when it comes to various visual relationship detection tasks. Developing on earlier tasks like object recognition, segmentation and captioning which focused on a relatively coarser image understanding, newer tasks have been introduced recently to deal with a finer level of image understanding. A Scene Graph is one such technique to better represent a scene and the various relationships present in it. With its wide number of applications in various tasks like Visual Question Answering, Semantic Image Retrieval, Image Generation, among many others, it has proved to be a useful tool for deeper and better visual relationship understanding. In this paper, we present a detailed survey on the various techniques for scene graph generation, their efficacy to represent visual relationships and how it has been used to solve various downstream tasks. We also attempt to analyze the various future directions in which the field might advance in the future. Being one of the first papers to give a detailed survey on this topic, we also hope to give a succinct introduction to scene graphs, and guide practitioners while developing approaches for their applications.
최근의 3D 대규모 멀티모달 모델(3D-LMM)은 복잡한 3D 장면 증거를 대규모 언어 모델(LLM)과 호환되는 제한된 수의 시각 토큰으로 압축하기 위해 시각 병목에 의존한다. 그러나 현재의 시각 병목은 이질적인 3D 증거를 동질적인 객체 중심 토큰 시퀀스로 수동적으로 압축하는 경우가 많아 장면의 공간적 조직이 충분히 표현되지 못한다. 이러한 과소 표현은 LLM이 평탄화된 토큰 시퀀스로부터 공간 관계를 복원하도록 강제하여, 관계 집약적이고 공간적으로 모호한 장면에서 불안정한 추론을 초래한다. 이 문제를 해결하기 위해 우리는 통합 3D 장면 이해를 위한 능동적 장면 상태 구성 프레임워크인 SceneScaffold를 제안한다. SceneScaffold는 시각 병목을 수동적 특징 압축기에서 능동적 장면 조직자로 재정식화하여, 언어 추론에 앞서 역할 인식 공간 스캐폴드를 구성한다. 구체적으로 SceneScaffold는 슈퍼포인트 수준의 시각 증거를 서로 다른 구조적 역할을 가진 장면 상태 구성요소로 조직한다. 엔티티 상태는 핵심 객체 의미를 보존하고, 장면 프레임 상태는 경계 및 영역 앵커를 통해 공간 참조를 유지하며, 관계 상태는 객체-환경 상호작용 단서를 인코딩하고, 전역 요약은 간결한 맥락을 제공한다. 이러한 역할 인식 구성을 통해 SceneScaffold는 언어 추론 전에 LLM에 공간적으로 조직된 장면 표현을 제공한다. 3D 시각적 접지, 질의응답, 밀집 캡셔닝을 포함한 통합 3D 장면 이해 과제에 대한 실험은 SceneScaffold의 효과를 입증하며, 진단 결과는 관계 집약적이고 공간적으로 모호한 사례에도 적용 가능함을 추가로 보여준다. 코드는 https://github.com/lixiangqi707/SceneScaffold 에서 이용할 수 있다.
While convolutional neural networks have gained impressive success recently in solving structured prediction problems such as semantic segmentation, it remains a challenge to differentiate individual object instances in the scene. Instance segmentation is very important in a variety of applications, such as autonomous driving, image captioning, and visual question answering. Techniques that combine large graphical models with low-level vision have been proposed to address this problem; however, we propose an end-to-end recurrent neural network (RNN) architecture with an attention mechanism to model a human-like counting process, and produce detailed instance segmentations. The network is jointly trained to sequentially produce regions of interest as well as a dominant object segmentation within each region. The proposed model achieves competitive results on the CVPPP, KITTI, and Cityscapes datasets.
Through a large-scale study over diverse face images, we show that facial attribute editing using modern generative AI models can severely degrade automated face recognition systems. This degradation persists even with identity-preserving generative models. To mitigate this issue, we propose two novel techniques for local and global attribute editing. We empirically ablate twenty-six facial semantic, demographic and expression-based attributes that have been edited using state-of-the-art generative models, and evaluate them using ArcFace and AdaFace matchers on CelebA, CelebAMaskHQ and LFW datasets. Finally, we use LLaVA, an emerging visual question-answering framework for attribute prediction to validate our editing techniques. Our methods outperform the current state-of-the-art at facial editing (BLIP, InstantID) while improving identity retention by a significant extent.
Scene Graph Generation (SGG) is a visual understanding task, aiming to describe a scene as a graph of entities and their relationships with each other. Existing works rely on location labels in form of bounding boxes or segmentation masks, increasing annotation costs and limiting dataset expansion. Recognizing that many applications do not require location data, we break this dependency and introduce location-free scene graph generation (LF-SGG). This new task aims at predicting instances of entities, as well as their relationships, without the explicit calculation of their spatial localization. To objectively evaluate the task, the predicted and ground truth scene graphs need to be compared. We solve this NP-hard problem through an efficient branching algorithm. Additionally, we design the first LF-SGG method, Pix2SG, using autoregressive sequence modeling. We demonstrate the effectiveness of our method on three scene graph generation datasets as well as two downstream tasks, image retrieval and visual question answering, and show that our approach is competitive to existing methods while not relying on location cues.
Visual Dialog (VD) is a task where an agent answers a series of image-related questions based on a multi-round dialog history. However, previous VD methods often treat the entire dialog history as a simple text input, disregarding the inherent conversational information flows at the round level. In this paper, we introduce Multi-round Dialogue State Tracking model (MDST), a framework that addresses this limitation by leveraging the dialogue state learned from dialog history to answer questions. MDST captures each round of dialog history, constructing internal dialogue state representations defined as 2-tuples of vision-language representations. These representations effectively ground the current question, enabling the generation of accurate answers. Experimental results on the VisDial v1.0 dataset demonstrate that MDST achieves a new state-of-the-art performance in generative setting. Furthermore, through a series of human studies, we validate the effectiveness of MDST in generating long, consistent, and human-like answers while consistently answering a series of questions correctly.
Programming based approaches to reasoning tasks have substantially expanded the types of questions models can answer about visual scenes. Yet on benchmark visual reasoning data, when models answer correctly, they produce incorrect programs 33% of the time. These models are often right for the wrong reasons and risk unexpected failures on new data. Unit tests play a foundational role in ensuring code correctness and could be used to repair such failures. We propose Visual Unit Testing (ViUniT), a framework to improve the reliability of visual programs by automatically generating unit tests. In our framework, a unit test is represented as a novel image and answer pair meant to verify the logical correctness of a program produced for a given query. Our method leverages a language model to create unit tests in the form of image descriptions and expected answers and image synthesis to produce corresponding images. We conduct a comprehensive analysis of what constitutes an effective visual unit test suite, exploring unit test generation, sampling strategies, image generation methods, and varying the number of programs and unit tests. Additionally, we introduce four applications of visual unit tests: best program selection, answer refusal, re-prompting, and unsupervised reward formulations for reinforcement learning. Experiments with two models across three datasets in visual question answering and image-text matching demonstrate that ViUniT improves model performance by 11.4%. Notably, it enables 7B open-source models to outperform gpt-4o-mini by an average of 7.7% and reduces the occurrence of programs that are correct for the wrong reasons by 40%.
Charts are common in literature across various scientific fields, conveying rich information easily accessible to readers. Current chart-related tasks focus on either chart perception that extracts information from the visual charts, or chart reasoning given the extracted data, e.g. in a tabular form. In this paper, we introduce StructChart, a novel framework that leverages Structured Triplet Representations (STR) to achieve a unified and label-efficient approach to chart perception and reasoning tasks, which is generally applicable to different downstream tasks, beyond the question-answering task as specifically studied in peer works. Specifically, StructChart first reformulates the chart data from the tubular form (linearized CSV) to STR, which can friendlily reduce the task gap between chart perception and reasoning. We then propose a Structuring Chart-oriented Representation Metric (SCRM) to quantitatively evaluate the chart perception task performance. To augment the training, we further explore the potential of Large Language Models (LLMs) to enhance the diversity in both chart visual style and statistical information. Extensive experiments on various chart-related tasks demonstrate the effectiveness and potential of a unified chart perception-reasoning paradigm to push the frontier of chart understanding.
While Multimodal Large Language Models (MLLMs) have achieved remarkable progress in open-ended visual question answering, they remain vulnerable to hallucinations. These are outputs that contradict or misrepresent input semantics, posing a critical challenge to the reliability and factual consistency. Existing methods often rely on external verification or post-hoc correction, lacking an internal mechanism to validate outputs directly during training. To bridge this gap, we propose ReLoop, a unified closed-loop training framework that encourages multimodal consistency for cross-modal understanding in MLLMs. ReLoop adopts a ring-shaped structure that integrates three complementary consistency feedback mechanisms, obliging MLLMs to "seeing twice and thinking backwards". Specifically, ReLoop employs the frozen Consistency Feedback Plugin (CFP), comprising semantic reconstruction, visual description, and an attention supervision module for attention alignment. These components collectively enforce semantic reversibility, visual consistency, and interpretable attention, enabling the model to correct its outputs during training. Extensive evaluations and analyses demonstrate the effectiveness of ReLoop in reducing hallucination rates across multiple benchmarks, establishing a robust method for hallucination mitigation in MLLMs. We will release our source code and data in the camera-ready version.
My goal in this paper is twofold: to study how well deep models can understand the images generated by DALL-E 2 and Midjourney, and to quantitatively evaluate these generative models. Two sets of generated images are collected for object recognition and visual question answering (VQA) tasks. On object recognition, the best model, out of 10 state-of-the-art object recognition models, achieves about 60\% and 80\% top-1 and top-5 accuracy, respectively. These numbers are much lower than the best accuracy on the ImageNet dataset (91\% and 99\%). On VQA, the OFA model scores 77.3\% on answering 241 binary questions across 50 images. This model scores 94.7\% on the binary VQA-v2 dataset. Humans are able to recognize the generated images and answer questions on them easily. We conclude that a) deep models struggle to understand the generated content, and may do better after fine-tuning, and b) there is a large distribution shift between the generated images and the real photographs. The distribution shift appears to be category-dependent. Data is available at: https://drive.google.com/file/d/1n2nCiaXtYJRRF2R73-LNE3zggeU_HeH0/view?usp=sharing.