All the latest news on Cybersecurity

What is the vulnerability? A zero-day security vulnerability has been uncovered in an enterprise file-transfer software CrushFTP. The vulnerability tagged as CVE-2024-4040 is actively being exploited in targeted attacks and has also been added to the CISA Known Exploited Vulnerabilities (KEV) list. The vulnerability allows unauthenticated remote attackers to read files from the file system outside of the VFS Sandbox, gain administrative access, and perform remote code execution on the server.What is the vendor Mitigation? According to the vendor advisory, CrushFTP versions prior to 10.7.1 and 11.1.0 are vulnerable to CVE-2024-4040 and being advised to immediately apply the patch. What FortiGuard Coverage is available? Endpoint vulnerability service is available to help detect vulnerable endpoints running the CrushFTP server application. FortiGuard Labs is further investigating for additional coverages.

Escape Vulnerability VFS Sandbox

Kaisen Linux is a distribution dedicated for IT professional based on Debian GNU/Linux. Large tools are integrated for diagnostics, rescue system and networks, lab creation and many more!

🔥 Bypass Kernel Anti-Cheat Here: https://guidedhacking.com/threads/how-to-bypass-kernel-anticheat-develop-drivers.11325/ 👨‍💻 Buy Our Courses: https://guidedhacking.com/register/ 💰 Donate on Patreon: https://patreon.com/guidedhacking ❤️ Follow us on Social Media: https://linktr.ee/guidedhacking ✏️ Tags: kernel hacks kernelmode anticheat guidedhacking Bypass Kernel Anti-Cheat reverse engineering kernel game hacking hacking games with kernel drivers kernel cheats vulnerable kernel drivers kernel game hacking bypass kernel drivers kernel cheats #gamehacking #kernel #anticheat Kernel Anti-Cheat Bypass

Bypass Kernel Driver Meme

For instance, from this graph, we can conclude that given the same time to complete the test, older users typically perform slightly better than their younger counterparts. This can be attributed to differences in visual experience and color perception needs across various age groups. You're invited to participate in the study by using the Dehancer Color Test. The second part of this series will focus on the algorithm used to calculate the number of colors based on the data we have gathered.

Dehancer Color Human Eye typically perform users typically

Remote paintings is turning into greater common in recent times, and it is essential to have the right techniques in location to hold matters jogging easily. I'm excited to share with you a few extremely good easy and effective ways to work collectively as a group, even when we're miles apart.

Easy Strategies hold matters jogging easily Remote Team Team Collaboration

:::info This paper is available on arxiv under CC 4.0 license. Authors: (1) Ryen W. White, Microsoft Research, Redmond, WA, USA. ::: Table of Links Abstract and Taking Search to task AI Copilots Challenges Opportunities The Undiscovered Country and References 5 THE UNDISCOVERED COUNTRY AI copilots will transform how we search. Tasks are central to people's lives and more support is needed for complex tasks in search settings. Some limited support for these tasks already exists in search engines, but copilots will expand the task frontier to make more tasks actionable and address the “last mile” in search interaction: task completion [58]. Moving forward, search providers should invest in “better together” experiences that utilize copilots plus traditional search, make these joint experiences more seamless for searchers, and add more support for their use in practice, e.g., help people to quickly understand copilot capabilities and potential and/or recommend the best modality for the current task or task stage. This includes experiences where both modalities are offered separately and can be selected by searchers and those where there is unification and the selection happens automatically based on the query and the conversation context. The foundation models that power copilots have other search-related applications, e.g., for generating and applying intent taxonomies [43] or for evaluation [19]. We must retain a continued focus on human-AI cooperation, where searchers stay in control while the degree of system support increases as needed [44], and on AI safety. Searchers need to be able to trust copilots in general but also be able to verify their answers with minimal effort. Overall, the future is bright for IR, and AI research in general, with the advent of generative AI and the copilots that build upon it. Copilots will help augment and empower searchers in their information seeking journeys. Computer science researchers and practitioners should embrace this new era of assistive agents and engage across the full spectrum of exciting practical and scientific opportunities, both within information seeking as we focused on here, and onwards into other important domains such as personal productivity [5] and scientific discovery [22]. REFERENCES [1] Eugene Agichtein, Ryen W White, Susan T Dumais, and Paul N Bennet. 2012. Search, interrupted: understanding and predicting search task continuation. In Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval. 315-324. [2] Marcia J Bates. 1990. Where should the person stop and the information search interface start? Information Processing & Management 26, 5 (1990), 575–591. [3] Nicholas J Belkin. 1980. Anomalous states of knowledge as a basis for information retrieval. Canadian journal of information science 5, 1 (1980), 133–143. [4] Paul N Bennett, Ryen W White, Wei Chu, Susan T Dumais, Peter Bailey, Fedor Borisyuk, and Xiaoyuan Cui. 2012. Modeling the impact of short-and long-term behavior on search personalization. In Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval. 185–194. [5] Christian Bird, Denae Ford, Thomas Zimmermann, Nicole Forsgren, Eirini Kalliamvakou, Travis Lowdermilk, and Idan Gazit. 2022. Taking Flight with Copilot: Early insights and opportunities of AI-powered pair-programming tools. Queue 20, 6 (2022), 35–57. [6] Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021). [7] Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. 2021. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP). IEEE, 141–159. [8] Andrei Broder. 2002. A taxonomy of web search. In ACM Sigir forum, Vol. 36. ACM New York, NY, USA, 3–10. [9] Andrei Z Broder and Preston McAfee. 2023. Delphic Costs and Benefits in Web Search: A utilitarian and historical analysis. arXiv preprint arXiv:2308.07525 (2023). [10] Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. 2023. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712 (2023). [11] Katriina Byström and Kalervo Järvelin. 1995. Task complexity affects information seeking and use. Information processing & management 31, 2 (1995), 191–213. [12] Robert Capra and Jaime Arguello. 2023. How does AI chat change search behaviors? arXiv preprint arXiv:2307.03826 (2023). [13] Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna. lmsys. org (accessed 14 April 2023) (2023). [14] Antonia Creswell and Murray Shanahan. 2022. Faithful reasoning using large language models. arXiv preprint arXiv:2208.14271 (2022). [15] Brenda Dervin. 1998. Sense-making theory and practice: An overview of user interests in knowledge seeking and use. Journal of knowledge management 2, 2 (1998), 36–46. [16] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018). [17] Karl Duncker and Lynne S Lees. 1945. On problem-solving. Psychological monographs 58, 5 (1945), i. [18] Brad Everman, Trevor Villwock, Dayuan Chen, Noe Soto, Oliver Zhang, and Ziliang Zong. 2023. Evaluating the Carbon Impact of Large Language Models at the Inference Stage. In 2023 IEEE International Performance, Computing, and Communications Conference (IPCCC). IEEE, 150–157. [19] Guglielmo Faggioli, Laura Dietz, Charles LA Clarke, Gianluca Demartini, Matthias Hagen, Claudia Hauff, Noriko Kando, Evangelos Kanoulas, Martin Potthast, Benno Stein, et al. 2023. Perspectives on Large Language Models for Relevance Judgment. In Proceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval. 39–50. [20] Jianfeng Gao, Chenyan Xiong, Paul Bennett, and Nick Craswell. 2023. Neural Approaches to Conversational Information Retrieval. Vol. 44. Springer Nature. [21] Ahmed Hassan Awadallah, Ryen W White, Patrick Pantel, Susan T Dumais, and Yi-Min Wang. 2014. Supporting complex search tasks. In Proceedings of the 23rd ACM international conference on conference on information and knowledge management. 829–838. [22] Tom Hope, Doug Downey, Daniel S Weld, Oren Etzioni, and Eric Horvitz. 2023. A computational inflection for scientific discovery. Commun. ACM 66, 8 (2023), 62–73. [23] Peter Ingwersen and Kalervo Järvelin. 2005. The turn: Integration of information seeking and retrieval in context. Vol. 18. Springer Science & Business Media. [24] Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. Comput. Surveys 55, 12 (2023), 1–38. [25] Thorsten Joachims. 2002. Optimizing search engines using clickthrough data. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining. 133–142. [26] Jeonghyun Kim. 2006. Task difficulty as a predictor and indicator of web searching interaction. In CHI'06 extended abstracts on human factors in computing systems. 959–964. [27] David R Krathwohl. 2002. A revision of Bloom's taxonomy: An overview. Theory into practice 41, 4 (2002), 212–218. [28] Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems 33 (2020), 9459–9474. [29] Yuelin Li and Nicholas J Belkin. 2008. A faceted approach to conceptualizing tasks in information seeking. Information processing & management 44, 6 (2008), 1822–1837. [30] Yuanchun Li and Oriana Riva. 2021. Glider: A reinforcement learning approach to extract UI scripts from websites. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1420–1430. [31] Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2021. Towards understanding and mitigating social biases in language models. In International Conference on Machine Learning. PMLR, 6565–6576. [32] Gary Marchionini. 2006. Exploratory search: from finding to understanding. Commun. ACM 49, 4 (2006), 41–46. [33] James Mayfield, Eugene Yang, Dawn Lawrie, Samuel Barham, Orion Weller, Marc Mason, Suraj Nair, and Scott Miller. 2023. Synthetic Cross-language Information Retrieval Training Data. arXiv preprint arXiv:2305.00331 (2023). [34] Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. 2023. Orca: Progressive learning from complex explanation traces of gpt-4. arXiv preprint arXiv:2306.02707 (2023). [35] Marc Najork. 2023. Generative Information Retrieval. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1–1. [36] Alexandra Olteanu, Jean Garcia-Gathright, Maarten de Rijke, Michael D Ekstrand, Adam Roegiest, Aldo Lipani, Alex Beutel, Alexandra Olteanu, Ana Lucic, AnaAndreea Stoica, et al. 2021. FACTS-IR: fairness, accountability, confidentiality, transparency, and safety in information retrieval. In ACM SIGIR Forum, Vol. 53. ACM New York, NY, USA, 20–43. [37] Soo Young Rieh, Kevyn Collins-Thompson, Preben Hansen, and Hye-Jung Lee. 2016. Towards searching as a learning process: A review of current perspectives and future directions. Journal of Information Science 42, 1 (2016), 19–34. [38] Shawon Sarkar and Chirag Shah. 2021. An integrated model of task, information needs, sources and uncertainty to design task-aware search systems. In Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval. 83–92. [39] Reijo Savolainen. 2012. Expectancy-value beliefs and information needs as motivators for task-based information seeking. Journal of Documentation 68, 4 (2012), 492–511. [40] Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761 (2023). [41] Chirag Shah. 2023. Generative AI and the Future of Information Access. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management (Birmingham, United Kingdom) (CIKM '23). Association for Computing Machinery, New York, NY, USA, 3. https://doi.org/10.1145/3583780.3615317 [42] Chirag Shah, Ryen White, Paul Thomas, Bhaskar Mitra, Shawon Sarkar, and Nicholas Belkin. 2023. Taking search to task. In Proceedings of the 2023 Conference on Human Information Interaction and Retrieval. 1–13. [43] Chirag Shah, Ryen W White, Reid Andersen, Georg Buscher, Scott Counts, Sarkar Snigdha Sarathi Das, Ali Montazer, Sathish Manivannan, Jennifer Neville, Xiaochuan Ni, et al. 2023. Using Large Language Models to Generate, Validate, and Apply User Intent Taxonomies. arXiv preprint arXiv:2309.13063 (2023). [44] Ben Shneiderman. 2022. Human-centered AI. Oxford University Press. [45] Adish Singla, Ryen White, and Jeff Huang. 2010. Studying trailfinding algorithms for enhanced web search. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval. 443–450. [46] Jaime Teevan, Kevyn Collins-Thompson, Ryen W White, and Susan Dumais. 2014. Slow search. Commun. ACM 57, 8 (2014), 36–38. [47] Jaime Teevan, Susan T Dumais, and Eric Horvitz. 2005. Personalizing search via automated analysis of interests and activities. In Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval. 449–456. [48] Jaime Teevan, Meredith Ringel Morris, and Steve Bush. 2009. Discovering and using groups to improve personalized search. In Proceedings of the second acm international conference on web search and data mining. 15–24. [49] Maartje ter Hoeve, Robert Sim, Elnaz Nouri, Adam Fourney, Maarten de Rijke, and Ryen W White. 2020. Conversations with documents: An exploration of document-centered assistance. In Proceedings of the 2020 Conference on Human Information Interaction and Retrieval. 43–52. [50] Paul Thomas, Seth Spielman, Nick Craswell, and Bhaskar Mitra. 2023. Large language models can accurately predict searcher preferences. arXiv preprint arXiv:2309.10621 (2023). [51] Randall H Trigg. 1988. Guided tours and tabletops: Tools for communicating in a hypertext environment. ACM Transactions on Information Systems (TOIS) 6, 4 (1988), 398–414. [52] Sarah K Tyler and Jaime Teevan. 2010. Large scale query log analysis of re-finding. In Proceedings of the third ACM international conference on Web search and data mining. 191–200. [53] Pertti Vakkari. 2001. A theory of the task-based information retrieval process: A summary and generalisation of a longitudinal study. Journal of documentation 57, 1 (2001), 44–60. [54] Pertti Vakkari. 2016. Searching as learning: A systematization based on literature. Journal of Information Science 42, 1 (2016), 7–18. [55] Nicholas Vincent. 2022. The Paradox of Reuse, Language Models Edition.https://nmvg.mataroa.blog/blog/the-paradox-of-reuse-language-modelsedition/. Accessed: 2023-09-12. [56] Yu Wang, Xiao Huang, and Ryen W White. 2013. Characterizing and supporting cross-device search tasks. In Proceedings of the sixth ACM international conference on Web search and data mining. 707–716. [57] Ryen W White. 2016. Interactions with search systems. Cambridge University Press. [58] Ryen W White. 2018. Opportunities and challenges in search interaction. Commun. ACM 61,12 (2018), 36–38. [59] Ryen W White. 2018. Skill discovery in virtual assistants. Commun. ACM 61, 11 (2018), 106–113. [60] Ryen W White. 2022. Intelligent futures in task assistance. Commun. ACM 65, 11 (2022), 35–39. [61] Ryen W. White. 2023. Tasks, Copilots, and the Future of Search. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (Taipei, Taiwan) (SIGIR '23). Association for Computing Machinery, New York, NY, USA, 5–6. https://doi.org/10.1145/3539618.3593069 [62] Ryen W White, Mikhail Bilenko, and Silviu Cucerzan. 2007. Studying the use of popular destinations to enhance web search interaction. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval. 159–166. [63] Ryen W White, Wei Chu, Ahmed Hassan, Xiaodong He, Yang Song, and Hongning Wang. 2013. Enhancing personalized search by mining and modeling task behavior. In Proceedings of the 22nd international conference on World Wide Web. 1411–1420. [64] Ryen W White, Adam Fourney, Allen Herring, Paul N Bennett, Nirupama Chandrasekaran, Robert Sim, Elnaz Nouri, and Mark J Encarnación. 2019. Multi-device digital assistance. Commun. ACM 62, 10 (2019), 28–31. [65] Ryen W White, Ian Ruthven, and Joemon M Jose. 2005. A study of factors affecting the utility of implicit relevance feedback. In Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval. 35–42. [66] Ryen W White, Ian Ruthven, Joemon M Jose, and CJ Van Rijsbergen. 2005. Evaluating implicit feedback models using searcher simulations. ACM Transactions on Information Systems (TOIS) 23, 3 (2005), 325–361. [67] Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. 2023. AutoGen: Enabling nextgen LLM applications via multi-agent conversation framework. arXiv preprint arXiv:2308.08155 (2023). [68] Iris Xie. 2008. Interactive information retrieval in digital environments. IGI global. [69] Jinyun Yan, Wei Chu, and Ryen W White. 2014. Cohort modeling for enhanced personalized search. In Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval. 505–514. [70] Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, et al. 2021. Differentially private fine-tuning of language models. arXiv preprint arXiv:2110.06500 (2021). [71] Hamed Zamani, Susan Dumais, Nick Craswell, Paul Bennett, and Gord Lueck. 2020. Generating clarifying questions for information retrieval. In Proceedings of the web conference 2020. 418–428. [72] Jieyu Zhang, Ranjay Krishna, Ahmed H Awadallah, and Chi Wang. 2023. EcoAssistant: Using LLM Assistant More Affordably and Accurately. arXiv preprint arXiv:2310.03046 (2023). [73] Yi Zhang, Sujay Kumar Jauhar, Julia Kiseleva, Ryen White, and Dan Roth. 2021. Learning to decompose and organize complex tasks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2726–2735. [74] Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Lingpeng Kong, Jiajun Chen, Lei Li, and Shujian Huang. 2023. Multilingual machine translation with large language models: Empirical results and analysis. arXiv preprint arXiv:2304.04675 (2023). [75] Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593 (2019).

ACM ACM SIGIR international ACM International Conference SIGIR conference

By Deeba Ahmed Hackers are dusting off old tricks! A recent attack exploited vulnerabilities in systems running outdates Microsoft Office to deliver Cobalt Strike malware. Learn how to protect yourself! This is a post from HackRead.com Read the original post: 7-Year-Old 0-Day in Microsoft Office Exploited to Drop Cobalt Strike

Drop Cobalt Office Exploited

Top cybersecurity companies combat cyberattacks and data breaches – From the editors at Cybercrime Magazine Sausalito, Calif. – Apr. 26, 2024 / IncidentResponders.com Looking for the top incident responders who tackle cyberattacks and data breaches? Look no further than our annual list. Before we get The post Who's Who In Incident Response appeared first on Cybercrime Magazine.

Magazine Sausalito

Rancher Modules This week, Metasploit community member h00die added the second of two modules targeting Rancher instances. These modules each leak sensitive information from vulnerable instances of the application which is intended to manage Kubernetes clusters. These are a great addition to Metasploit's coverage for testing Kubernetes environments. PAN-OS

Rancher Rancher Modules

:::info This paper is available on arxiv under CC 4.0 license. Authors: (1) Ryen W. White, Microsoft Research, Redmond, WA, USA. ::: Table of Links Abstract and Taking Search to task AI Copilots Challenges Opportunities The Undiscovered Country and References 4 OPPORTUNITIES For some time, scholars have argued that the future of information access will involve personal search assistants with advanced capabilities, including natural language input, rich sensing, user/task/world models, and reactive and proactive experiences [57]. Technology is catching up with this vision. Opportunities going forward can be grouped into four areas: (1) Model innovation; (2) Next-generation experiences; (3) Measurement, and; (4) Broader implications. The opportunities are summarized in Figure 6. There are likely more such opportunities that are not listed here, but the long list shown in the figure is a reasonable starting point for the research community. 4.1 Model Innovation There are many opportunities to better model search situations and augment and adapt foundational models to better align with searchers' tasks and goals, and provide more accurate answers. Copilots can leverage these model enhancements to improve the support that they provide for complex search tasks. 4.1.1 Task modeling. Opportunity: Build richer task models that more fully represent tasks and task contexts. This includes how we infer tasks (e.g., from textual content of search process, from usersystem interactions, from other situational and contextual information such as location, time, and application usage) and how we represent those tasks internally (e.g., as a hierarchy (Figure 1) or a more abstract representation (semantic vectors, graph embeddings, Markov models, and so on)). We also need to be able to estimate key task characteristics, such as task complexity, which, in one use, can help search systems route requests to the most appropriate modality. In addition, we need to find ways for copilots to collect more user/world knowledge, both in general and specifically related to the task at hand. A better understanding of the task context will help copilots more accurately model the tasks themselves. 4.1.2 Alignment. Opportunity: Develop methods to continuously align copilots to tasks/goals/values via feedback, e.g., conversation content as feedback (e.g., searchers expressing gratitude to the copilot in natural language) or explicit feedback on copilot answers via likes and dislikes. The performance of copilots that are missing alignment will remain fixed over time. Copilots need applicationaligned feedback loops to better understand searcher goals and tasks and use that feedback to continuously improve answer accuracy and relevance. Beyond research on fine-tuning foundation models from human feedback (e.g., likes/dislikes) [75], we can also build on learnings from research on implicit feedback in IR, including work on improving ranking algorithms via SERP clicks [25] and developing specialized interfaces to capture user feedback [65]. 4.1.3 Augmentation. Opportunity: Augment copilots with relevant external knowledge and enhanced tools and capabilities. As mentioned earlier, RAG is a common form of knowledge injection for foundation models. Relevance models are tuned to maximize user benefit, not for copilot consumption. We need to evaluate whether this difference is meaningful practically and if so, develop new ranking criteria that consider the intended consumer of the search results (human or machine). Despite their incredible capabilities, foundation models still have shortcomings that manifest in the copilots that use them. We need to understand these shortcomings through evaluation and find ways to leverage external skills/plugins to address them. Copilots must find and recommend skills per task demands [59], e.g., invoking Wolfram for computational assistance. We can also integrate tool use directly into tool-augmented models, e.g., Toolformer [40], that can teach themselves to use tools. Models of task context may also be incomplete and we should invest in ways to better ground copilot responses via context, e.g., richer sensing, context filtering, and dynamic prompting. 4.1.4 Grounding. Opportunity: Use grounding to reduce hallucinations, build searcher trust, and support content creators. It is in the interests of copilots, searchers, and content creators (and providers and advertisers) to consider the source of the data used in generating answers. Provenance is critical and copilots should provide links back to relevant sources (preferably with specific details/URLs not generalities/domains) to help build user trust, provide attribution for content creators, and drive engagement for content providers and advertisers. It also important for building trust and for supporting learning for copilots to practice faithful reasoning [14], and provide intepretable reasoning traces (e.g., explanations with chain-of-thought) associated with their answers. We should also think about how we integrate search within existing experiences (e.g., in other copilots) to ground answers in their context of use and in more places that people seek those answers. 4.1.5 Personalization. Opportunity: Develop personal copilots that understand searchers and their tasks, using personal data, privately and securely. Searchers bring their personal tasks to search systems and copilots will be no different. Here are some example personal prompts that describe the types of personal tasks that searchers might expect a copilot to handle: (1) Write an e-mail to my client in my personal style with a description of the quote in the attached doc. (2) Tell me what's important for me to know about the company town hall that I missed? (3) Where should I go for lunch today? These tasks span creation, summarization, and recommendation and quickly illustrate the wide range of expectations that people may have from their personal copilots. As part of developing such personalized AI support, we need to: (1) Study foundation model capabilities, including their ability to identify task-relevant information in personal data and activity histories, and model user knowledge in the current task and topic, and (2) Develop core technologies, including infinite memory, using relevant long-term activity (in IR, there has been considerable research on relevant areas such as re-finding [52] and personalization [47]); context compression, to fit more context into finite token limits (e.g., using turn-by-turn summarization rather than raw conversational content); privacy, including mitigations such as differential privacy and federated learning, and research on machine unlearning [7] to intentionally forget irrelevant information over time, including sensitive information that the searcher may have explicitly asked to be removed from the foundation model. 4.1.6 Adaptation. Two main forms of adaptation that we consider here are model specialization and so-called adaptive computation. • Model specialization. Opportunity: Develop specialized foundation models for search tasks that are controllable and efficient. Large foundation models are generalists and have a wide capability surface. Specializing these models for specific tasks and applications discards useless knowledge, making the models more accurate and efficient for the task at hand. Recent advances in this area have yielded strong performance, e.g., the Orca-13B model [34] uses explanation-based tuning (where the model explains the steps used to achieve its output and those explanations are used to train a small language model) to outperform state-of-the-art models of a similar size such as Vicuna-13B [13]. Future work could explore guiding specialization via search data, including anonymized large-scale search logs, and as well as algorithmic advances in preference modeling and continual learning. • Adaptive computation. Opportunity: Develop methods to adaptively apply different models per task and application demands. Adaptive compute involves using multiple foundation models (e.g., GPT-4 and a specialized model) each with different inferencetime constraints, primarily around speed, capabilities, and cost, and learning which model to apply for a given task. The specialized model can backoff to one or more larger models as needed per task demands. The input can be the task plus the constraints of the application scenario under which the model must operate. Human feedback on the output can also be used to improve model performance over time [72]. These adaptation methods will yield more effective and more efficient AI capabilities that copilots can use to help searchers across a range of settings, including in offline settings (e.g., on-device only). 4.2 Next-Generation Experiences Advancing models is necessary but not sufficient given the central role that interaction plays in the search process [57]. There are many opportunities to develop new search experiences that capitalize on copilot capabilities while keeping searchers in control. 4.2.1 Search + Copilots. Opportunity: Develop experiences bridging the search and copilot (chat) modalities, offering explanations and suggestions. Given how entrenched and popular traditional search is, it is likely that some form of query-result interaction will remain a core part of how we find information online. Future, copilot-enhanced experiences may reflect a more seamless combination of the two modalities in a unified experience. Both Google and Bing are taking a step in that direction by unifying search results and copilot answers in a single interface. Explanations on what each modality and style (e.g., creative, balanced, and precise) are best for will help searchers make decisions about which modalities and settings to use and when. Modality recommendation given task is also worth exploring: simple tasks may only need traditional search, whereas complex tasks may need copilots. Related to this are opportunities around conversation style suggestion given the current task, e.g., fact-finding task or short reply (needs precision) and generating new content (needs creativity). Search providers could also consider offering a single point of entry and an automatic routing mechanism to direct requests to the correct modality given inferences about the underlying task (e.g., from Section 4.1.1) and the appropriateness of each of the modalities for that task. 4.2.2 Human Learning. Opportunity: Develop copilots that can detect learning tasks and support relevant learning activities. As mentioned earlier, copilots can remove or change human learning opportunities by their automated generation and provision of answers. Learning is a core outcome of information seeking [15, 32, 54]. We need to develop copilots that can detect learning and sensemaking tasks, and support relevant learning activities via copilot experiences that, for example, provide detailed explanations and reasoning, offer links to learning resources (e.g., instructional videos), enable deep engagement with task content (e.g., via relevant sources), and support specifying and attaining learning objectives. 4.2.3 Human Control. Opportunity: Better understand control and develop copilots with control while growing automation. Control is an essential aspect of searcher interaction with copilots. Copilots should consult humans to resolve or codify value tensions. Copilots should be in collaboration mode by default and must only take control with the permission of stakeholders. Experiences that provide searchers with more agency are critical, e.g., adjust specificity/diversity in copilot answers, leading to less generality and less repetition. As mentioned in Section 4.1.4, citations in answers are important. Humans need to be able to verify citation correctness in a lightweight way, ideally without leaving the user experience. We also need a set of user studies to understand the implications of less control of some aspects (e.g., answer generation), more control over other aspects (e.g., macrotask specification), and control over new aspects, such as conversation style and tone. 4.2.4 Completion. Opportunity: Copilots should help searchers complete tasks while keeping searchers in control. We need to both expand the task frontier by adding/discovering more capabilities of foundation models that can be surfaced through copilots and deepen task capabilities so that copilots can help searchers better complete more tasks. We can view skills and plugins as actuators of the digital world and we should help foundation models fully utilize them. We need to start simple (e.g., reservations), learn and iterate, and increase task complexity as model capabilities improve with time. The standard mode of engagement with copilots is reactive; searchers send requests and the copilots respond. Copilots can also take initiative, with permission, and provide updates (for standing tasks) and proactive suggestions to assist the searcher. Copilots can also help support task planning for complex tasks such as travel or events. AI can already help complete repetitive tasks, e.g., action transformers, trained on digital tools[8] or create and apply “tasklets” (user interface scripts) learned from websites [30]. Given the centrality of search interaction in the information seeking process, it is important to focus sufficient attention on interaction models and experiences in copilots. In doing so, we must also carefully consider the implications of critical decisions on issues that affect AI in general such as control and automation. 4.3 Measurement Another important direction is in measuring copilot performance, understanding copilot impact and capabilities, and tracking copilot evolution over time. Many of the challenges and opportunities in this area also affect the evaluation of foundation models in general (e.g., non-determinism, saturated benchmarks, inadequate metrics). 4.3.1 Evaluation. Opportunity: Identify and develop metrics for copilot evaluation, while considering important factors, and find applications of copilot components for IR evaluation. There are many options for copilot metrics, including feedback, engagement, precisionrecall, generation quality, answer accuracy, and so on. Given the task focus, metrics should likely target the task holistically (e.g., success, effort, satisfaction). In evaluating search copilots, it is also important to consider: (1) Repeatability: Non-determinism can make copilots difficult to evaluate/debug; (2) Interplay between search and copilots (switching, joint task success, etc.); (3) Longer term effects on user capabilities and productivity; (4) Task characteristics: Complexity, etc., and; (5) New benchmarks: Copilots affected by external data, grounding, queries, etc. There are also opportunities to consider applications of copilot components for IR evaluation. Foundation models can predict searcher preferences [50] and assist with relevance judgments [19], including generating explanations for judges. Also, foundation models can create powerful searcher simulations that can better mimic human behavior and values, and expand on early work on searcher simulations in IR [66]. 4.3.2 Understanding. Opportunity: Deeply understand copilot capabilities and copilot impact on searchers and on their tasks. We have only scratched the surface in understanding the copilots and their effects. A deeper understanding takes a few forms, including: (1) User understanding: Covering mental models of copilots and effects of bias (e.g., functional fixedness [17]) on how copilots are adopted and used in search settings. It also covers changes in search behavior and information seeking strategies, including measuring changes in effects across modalities, e.g., search versus copilots and search plus copilots. There are also opportunities in using foundation models to understand search interactions via user studies [12] and use foundation models to generate intent taxonomies and classify intents from log data [43]; (2) Task understanding: Covering the intents and tasks that copilots are used for and most effective for, and; (3) Copilot understanding: Covering the capabilities and limitations of copilots, e.g., similar to the recent “Sparks of AGI” paper on GPT-4 [10], which examined foundation model capabilities. Measuring copilot performance is essential in understanding their utility and improving their performance over time. Copilots do not exist in a vacuum and we must consider the broader implications of their deployment for complex tasks in search settings. 4.4 Broader Implications Copilots must function in a complex and dynamic world. There are several opportunities beyond advances in technology and in deepening our understanding of copilot performance and capabilities. 4.4.1 Responsibility. Opportunity: Understand factors affecting reliability, safety, fairness, and inclusion in copilot usage. The broad reach of search engines means that copilots have an obligation to act responsibly. Research is needed on ways to understand and improve answer accuracy via better grounding in more reliable data sources, develop guardrails, understand biases in foundation models, prompts, and the data used for grounding, and understand how well copilots work in different contexts, with different tasks, and with different people/cohorts. Red teaming, user testing, and feedback loops are all needed to determine emerging risks in copilots and the foundation models that underlie them. This also builds on existing work on responsible AI and responsible IR and FACTS-IR, which has studied biases and harms, and ways to mitigate them [36]. 4.4.2 Economics. Opportunity: Understand and expand the economic impact of copilots. This includes exploring new business models which copilots will create beyond information finding. Expanding the task frontier from information finding deeper into task completion (e.g., into creation and analysis) creates new business opportunity. It also unlocks new opportunities for advertising, including advertisements that are shown inline with dialog/answers and contextually relevant to the current conversation. There is also a need to more deeply understand the impact of copilots on content creation and search engine optimization. Content attribution is vital in such scenarios to ensure that content creators (and advertisers and publishers) can still generate returns. We should avoid the so-called “paradox of reuse” [55] where lower visits to online content leads to less content being created which in turn leads to worse models over time. Another important aspect of economics is the cost-benefit trade-off and is related to work on adaptation (Section 4.1.6). Large model inference is expensive and unnecessary for many applications. This cost will reduce with optimization, for which model specialization and adaptive computation can help. 4.4.3 Ubiquity. Opportunity: Copilot integrations to model and support complex search tasks. Copilots must co-exist with the other parts of the application ecosystem. Search copilots can be integrated into applications such as Web browsers (offering in-browser chat, editing assistance, summarization) and productivity applications (offering support in creating documents, emails, presentations, etc.). These copilots can capitalize on application context to do a better job of answering searcher requests. Copilots can also span surfaces/applications through integration with the operating system. This enables richer task modeling and complex task support, since such tasks often involve multiple applications. Critically, we must do this privately and securely to mitigate risks for copilot users. 4.5 Summary The directions highlighted in this section are just examples of the opportunities afforded by the emergence of generative AI and copilots in search settings. There are other areas for search providers to consider too, such as multilingual copilot experiences (i.e., foundation models are powerful and could help with language translation [33, 74]), copilot efficiency (i.e., large model inference is expensive and not sustainable at massive scale, so creative solutions are needed [72]), the carbon impact from running foundation models at scale to serve billions of answers for copilots [18], making copilots private by design [70], and government directives (e.g., the recent executive order from U.S. President Biden on AI safety and security[9]) and legislation, among many other opportunities. [8] https://www.adept.ai/blog/act-1 [9] https://www.whitehouse.gov/briefing-room/statements-releases/2023/10/30/factsheet-president-biden-issues-executive-order-on-safe-secure-and-trustworthyartificial-intelligence/

:::info This paper is available on arxiv under CC 4.0 license. Authors: (1) Ryen W. White, Microsoft Research, Redmond, WA, USA. ::: Table of Links Abstract and Taking Search to task AI Copilots Challenges Opportunities The Undiscovered Country and References 3 CHALLENGES Despite the promise of copilots, there are significant challenges that should be acknowledged and we must find ways to overcome. Those include issues with the copilot output shown in response to searcher requests, the impacts that the copilots can have on searchers, and shifts in the degree of agency that humans have in the search process that result from the introduction of copilots. • Hallucinations: Searchers rely a lot on the answers from copilots,but those answers can be erroneous or non-sensical. So-called“hallucination” is a well-studied problem in foundation models [24]. Copilots can hallucinate for many reasons. One of the main reasons being gaps in the training data. RAG, discussed earlier, is a way to help address this by ensuring that the copilot has access to up-to-date, relevant information at inference time to help ground its responses. Injection of knowledge from other external sources, such as knowledge graphs and Wikipedia, can also help improve the accuracy of copilot responses. An issue related to copilots surfacing misinformation is toxicity (i.e., offensive or harmful content), which can also be present in the copilot output, and must be mitigated before answers are shown to searchers. • Biases: Biases in the training data, e.g., social biases and stereotypes [31], affect the output of foundation models and hence the answers provided by copilots. Synthesis of content from different sources can amplify biases in this data. As with hallucinations, this is a well-studied problem [6]. Copilots are also subject to biases from learning from their own or other AI generated content (via feedback loops); biased historical sequences lead to biased downstream models. Copilots may also amplify existing cognitive biases, such as confirmation bias, by favoring responses that are aligned with searchers' existing beliefs and values, and by providing responses that keep searchers engaged with the copilot, regardless of the ramifications for the searcher. • Human learning: Learning may be affected/interrupted by the use of AI copilots since they remove the need for searchers to engage as fully with the search system and the information retrieved. Learning is already a core part of the search process [32, 37, 54]. Both exploratory search and search as learning involve considerable time and effort in finding and examining relevant content. While this could be viewed as a cost, this deep exposure to content also helps people learn. As mentioned earlier, copilot users can ask richer questions (allowing them to specify their tasks and goals more fully) but they then receive synthesized answers generated by the copilot, creating fewer, new, or simply different learning opportunities for humans that must be understood. • Human control: Supporting search requires considering the degree of searcher involvement in the search process, which varies depending on the search task [2]. Copilots enable more strategic, higher-order actions (higher up the “task tree” from Figure 1 than typical interactions with search systems). It is clear that searchers want control over the search process. They want to know what information is/not being included and why. This helps them understand and trust system output. As things stand, copilot users delegate full control of answer generation to the AI, but the rest is mixed, i.e., less control of search mechanics (queries, etc.) but more control of task specifications (via natural language and dialog). There is more than just a basic tension between automation and control. In reality, it is not a zero sum game. Designers of copilots need to ensure human control while increasing automation [44]. New frameworks for task completion are moving in this direction. For example, AutoGen [67], uses multiple specialized assistive AI copilots that engage with humans and with each other directly to help complete complex tasks, with humans staying informed and in control throughout. Overall, these are just a few of the challenges that affect the viability of copilots. There are other challenges, such as deeply ingrained search habits that may be a barrier to the adoption of new search functionality, despite the clear benefits to searchers.

searchers

Telegram users are currently experiencing issues worldwide, with users unable to use the website and mobile apps. [...]

experiencing issues users unable