Color vision deficiency(CVD) is a disability that affects millions of people in the world and that has an effect on the perceiver of color when it comes to distinguishing colors in their daily activities like drawing clothes and differentiating objects. Although the recent developments in computer vision made it possible to apply the assistive technology to color-blind users, many of the existing systems rely on simple color perception and do not customize and provide context-sensitive input. As introduced in this paper, ColorInsightX is an AI-based assistive system that offers convenience to people with CVD in terms of accessibility. The proposed system will combine the Farnsworth Munsell 100 Hue Test to identify the approach of the type of color vision deficiency and confusion axis of the user, ZOLO-Tiny object detector which is also based on the YOLO architecture, and color extraction algorithms to identify the dominant colors. As a visual accessibility measure, the system also has Daltonization based color correction that produces custom visuals depending on the users color perception profile. This system is developed with the help of react native in the front-end and FastAI in the back-end so that color recognition and object-detection via camera can be done in real time. There are also text-to-speech feedback which gives audio feedback on objects and colors found and a rule-based outfit suggestion module which will aid in the user choosing outfits that are notable. ColorInsightX provides a context-dependent, individualized, and multi- modal assistive technology as compared to the conventional color recognition applications; this enablespeople with color vision deficiency to gain greater independence and usability