A NOVEL TDEF1.0 FOR MAKING TWITTER
ACCESSIBLE FOR PEOPLE WITH
DISABILITIES
Namrata Aggarwal
Dept. of Computer Science and Applications. Bharati Vidyapeeth’s Institute of
Computer Applications and Management (BVICAM). Delhi, India
aggarwalnamrata644@gmail.com
Ritika Wason*
Dept. of Computer Science and Applications. Bharati Vidyapeeth’s Institute of
Computer Applications and Management (BVICAM). Delhi, India
ritika.wason@bvicam.in
Parul Arora
Dept. of Computer Science and Applications. Bharati Vidyapeeth’s Institute of
Computer Applications and Management (BVICAM). Delhi, India
paruldevsum@gmail.com
Aruna Tomar
Dept. of Computer Science and Applications. Bharati Vidyapeeth’s Institute of
Computer Applications and Management (BVICAM). Delhi, India
tomar07aruna@gmail.com
Devansh Arora
Indraprastha Institute of Information Technology (IIIT). Delhi, India
devansh2005@iiitd.ac.in
Reception: 20/02/2023 Acceptance: 21/04/2023 Publication: 10/07/2023
Suggested citation:
Aggarwal, N., Wason, R., Arora, P., Tomar, A. and Arora, D. (2023). A Novel
TDEF1.0 for Making Twitter Accessible for People with Disabilities. 3C
Tecnología. Glosas de innovación aplicada a la pyme, 12(2), 31-47. https://
doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.44 | Iss.12 | N.2 April - June 2023
31
ABSTRACT
This manuscript introduces a novel framework to extend the accessibility of Twitter
users’ timelines to people with disabilities. Our proposed framework is designed with
iconic speaker and information functionalities which will enable transcription of multi-
media content and provide users the opportunity to read and hear the translated
transcripts depending upon the user’s primary language.
This work is one of its kind that opens Twitter’s user timeline completely to people with
disabilities.
KEYWORDS
Disabled people, Accessibility, Twitter, Transcription, TDEF1.0.
INDEX
ABSTRACT
KEYWORDS
1. TWITTER FOR EVERYONE
2. AN ANALYSIS OF THE CURRENT TWITTER FRAMEWORK
2.1. Current Timeline features
3. PROPOSED TWITTER DATA EXTRACTION FRAMEWORK (TDEF 1.0)
3.1. Voice Tweet
3.2. Video Tweet
3.3. Image/Gif Tweet
3.4. Text Tweet
4. IMPLEMENTATION DETAILS
4.1. Extraction of Tweets
4.2. Transcription
4.3. Image Captioning
4.4. Text Summarization
5. RESULTS
6. CONCLUSION
7. FUTURE SCOPE
REFERENCES
ABOUT THE AUTHORS
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.44 | Iss.12 | N.2 April - June 2023
32
ABSTRACT
This manuscript introduces a novel framework to extend the accessibility of Twitter
users’ timelines to people with disabilities. Our proposed framework is designed with
iconic speaker and information functionalities which will enable transcription of multi-
media content and provide users the opportunity to read and hear the translated
transcripts depending upon the user’s primary language.
This work is one of its kind that opens Twitter’s user timeline completely to people with
disabilities.
KEYWORDS
Disabled people, Accessibility, Twitter, Transcription, TDEF1.0.
INDEX
ABSTRACT
KEYWORDS
1. TWITTER FOR EVERYONE
2. AN ANALYSIS OF THE CURRENT TWITTER FRAMEWORK
2.1. Current Timeline features
3. PROPOSED TWITTER DATA EXTRACTION FRAMEWORK (TDEF 1.0)
3.1. Voice Tweet
3.2. Video Tweet
3.3. Image/Gif Tweet
3.4. Text Tweet
4. IMPLEMENTATION DETAILS
4.1. Extraction of Tweets
4.2. Transcription
4.3. Image Captioning
4.4. Text Summarization
5. RESULTS
6. CONCLUSION
7. FUTURE SCOPE
REFERENCES
ABOUT THE AUTHORS
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
1. TWITTER FOR EVERYONE
Among the various social media platforms [8], Twitter is considered one of the
popular microblogging services on the internet which allow users to post (tweet), like,
comment and share (retweet) [2].
Twitter has over 3.5 billion monthly active users [4],[18]. This number is growing
each day as Twitter is continuously improving its global accessibility to share real-time
information among members of the public and is thus widely impacting business,
politics, communities, social groups, etc [10],[18].
At the basic, Twitter makes its features accessible to all its users with options to
create content in audio, video, text, image, or URL formats [3]. This feature is included
with the purpose to widen the outreach of Twitter content among a wide audience
including the blind or people with vision abnormalities [20].
Twitter being an internationally recognizable platform, follows a format for
appropriate appearances and limited functionality [3]. It also introduced certain
features to improve its content accessibility, like [21].
1. In 2017, the limit of text-tweet got doubled from 140-character-limitation to 280.
2. As of May 2020, Alt-Text for captioning images to improve the accessibility of
Twitter timelines for visually impaired people is enabled by default on Web, iOS
and Android app users [5].
3. Voice tweets were rolled out as an experiment, which was limited to iOS
devices to improve the accessibility of content to the audience [29].
However, as per analysis, the new Twitter features are either extended versions of
existing features or are limited to several users [25].
Hence, though efforts are being made to open the boundaries of Twitter to people
with disabilities, challenges remain [17],[21].
This manuscript proposes a novel solution to widen the feature functionality of
Twitter, ensuring easy access to all potential users with varied types and degrees of
disability.
The manuscript proposes to increase the participation of people with disability
through the inculcation of novel features like text-to-speech (Read out loud tweet),
image captioning (Image summarization), speech-to-text (Video/Audio tweet
captioning), etc. To highlight the same, the rest of this manuscript is structured as
follows. Section ii analyses the existing Twitter framework concerning the timeline
features. Section iii outlines the proposed Twitter Data Extraction Framework (TDEF).
Section iv details the implementation details of the same. Section v reports the results
of testing the same while Sections vi and vii conclude the manuscript by elaborating
our understanding of the same and our future course of action to address further
research challenges.
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.44 | Iss.12 | N.2 April - June 2023
33
2. AN ANALYSIS OF THE CURRENT TWITTER
FRAMEWORK
Twitter is one of the most popular global social networking platforms [4]. Hence
before suggesting any enhancements to its features, it is first important to understand
its current capabilities [8]. We review the same in this section, especially concerning
the timeline feature.
2.1. CURRENT TIMELINE FEATURES
Twitter displays a structured timeline with a stream of real-time tweets from the
accounts that are followed by any user [18]. It also provides options to view the top
tweets or the latest tweets first in the timeline [2]. The major goal of the user timeline
is to display the content the user is most interested in and would contribute to the
same [7].
As per their most popular historical features –comments and retweets made a huge
success in the engagement of potential users with the content [3]. To further improve
accessibility, recently the timeline got an extension with follow-up features, like [21]:
1. Anonymous Bookmark [26]: Introduced to avoid the problem of liking,
retweeting and spamming. Enabled users to purposely refer to the tweets later.
Bookmarked content remains private as no one can see who saved and what
is saved.
2. Direct Message (DM) [11]: Allows private sharing of tweets through DM (Direct
message) or any social platform. It applies the copy link feature to copy the
URL into the clipboard. The reach of tweets is now amplified to varied
platforms like SMS, Email, etc outside of Twitter to stabilize its online presence.
3. Twitter Fleets [23]: Enables sharing transitory thoughts through your tweet,
text, videos, gifs, and photos for 24 hours. Inspired by Instagram and
Facebook, it is a dynamic, personalized approach rolled out to share moments
for a short period and see who viewed the content.
4. Voice Tweets [16],[27]: Enabled tweets to be published with audio options that
people could play.
The above-listed features are only a few of the most notable ones [12],[16].
However, we noticed that the Alt-text option, which enabled limited image description
for visually impaired people expanded the number of potentially active users on
Twitter [12],[22],[24] Hence, the current updated timeline of Twitter supporting people
with disabilities is detailed in Table 1 below:
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.44 | Iss.12 | N.2 April - June 2023
34
2. AN ANALYSIS OF THE CURRENT TWITTER
FRAMEWORK
Twitter is one of the most popular global social networking platforms [4]. Hence
before suggesting any enhancements to its features, it is first important to understand
its current capabilities [8]. We review the same in this section, especially concerning
the timeline feature.
2.1. CURRENT TIMELINE FEATURES
Twitter displays a structured timeline with a stream of real-time tweets from the
accounts that are followed by any user [18]. It also provides options to view the top
tweets or the latest tweets first in the timeline [2]. The major goal of the user timeline
is to display the content the user is most interested in and would contribute to the
same [7].
As per their most popular historical features –comments and retweets made a huge
success in the engagement of potential users with the content [3]. To further improve
accessibility, recently the timeline got an extension with follow-up features, like [21]:
1. Anonymous Bookmark [26]: Introduced to avoid the problem of liking,
retweeting and spamming. Enabled users to purposely refer to the tweets later.
Bookmarked content remains private as no one can see who saved and what
is saved.
2. Direct Message (DM) [11]: Allows private sharing of tweets through DM (Direct
message) or any social platform. It applies the copy link feature to copy the
URL into the clipboard. The reach of tweets is now amplified to varied
platforms like SMS, Email, etc outside of Twitter to stabilize its online presence.
3. Twitter Fleets [23]: Enables sharing transitory thoughts through your tweet,
text, videos, gifs, and photos for 24 hours. Inspired by Instagram and
Facebook, it is a dynamic, personalized approach rolled out to share moments
for a short period and see who viewed the content.
4. Voice Tweets [16],[27]: Enabled tweets to be published with audio options that
people could play.
The above-listed features are only a few of the most notable ones [12],[16].
However, we noticed that the Alt-text option, which enabled limited image description
for visually impaired people expanded the number of potentially active users on
Twitter [12],[22],[24] Hence, the current updated timeline of Twitter supporting people
with disabilities is detailed in Table 1 below:
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
Table 1. Twitter support for people with disabilities
As is clear from Table 1, for people with disabilities, Twitter has made some inbuilt
support [24]
. For multi-lingual users, auto-translation for text tweets is also rolled out
powered by Google to help users to engage with translated content in a meaningful
way [16]. The same is depicted in Fig.1 below:
Figure 1. A tweet featuring Auto-translation into the user’s primary language
Popular platforms like Facebook and LinkedIn have already implemented multiple
features to increase the ability to access content like text, voice, video image/gif [10].
Thus, it is time Twitter should keep in mind that Twitter is for everyone and it is a room
where potential users engage with content and raise their voice [21]
. Thus, as one of
its first, we have designed and implemented a novel Twitter Data Extraction
Framework we have coined as TDEF1.0. This framework is intended to support
Disability Voice
Video with
Speech
Image Text
Visual (Blind/
Colour Blind/
Visually
Impaired)
-Under Trial Alt-Text
(optional)
Alt-Text
(optional)
Voice (Deaf/
Hearing
Imapired)
Under Trial Under Trial - -
Reading
(Dyslexia)
----
Language (non-
natives) ---
Text Translation
(through
Google)
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.44 | Iss.12 | N.2 April - June 2023
35
people with disabilities to join and use Twitter. The details of the same are elaborated
in the following section.
3. PROPOSED TWITTER DATA EXTRACTION
FRAMEWORK (TDEF 1.0)
As discussed in section ii, Twitter has various accessibility features for normal
people [11],[12],[21],[23],[26],[27],[14],[18]
. However, it is well realized that Twitter
should improve its accessibility among physically challenged prospective users [17].
In the past for many years, Twitter limited its use to traditional features for content
creation [13]
. However, soon it began rolling out effective and engaging features to
allow more users to interact with real-time information in a more meaningful way [21].
In 2015, Twitter introduced many extensions to its timeline to leverage usage and
increase the number of active users [18]
. However, after thorough timeline analysis
we realized that despite efforts, Twitter has as yet failed to introduce any timeline
enhancements that could bring people with disabilities easily on board. Though many
disabled have been absorbed by Twitter, many still hesitate [17].
Thus, we propose a real-time approach to include disabled people on Twitter.
Taking cue [15]
and amplifying Twitter values for the people, our research majorly
focussed on the accessibility of Voice, Video, Images and Text to support people with
different challenges and allow them to express and present their opinions without any
technical barriers. We now briefly explain our approach for different kinds of tweets:
3.1. VOICE TWEET
After Twitter’s recent experiment of Voice tweets for the iOS app [11], users could
tweet their voice using the voice icon. Users could initially record up to 140 seconds of
audio. If the audio message was longer than 140 seconds, it will be automatically
threaded up to 25 audio tweets. We realized that the feature could be made more
inclusive for the disabled community through transcription and translation.
Through transcription, the user could read the text from the transcribed audio.
Through automated translation captions, users could also read the text in different
languages which would make audio tweets accessible to the hearing-impaired
audience.
3.2. VIDEO TWEET
To make video tweets accessible, we performed operations to generate a vocal
summary of video tweets with speech and without speech. Summarized text will help
the users to engage with content in a short period in more customized ways.
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.44 | Iss.12 | N.2 April - June 2023
36
people with disabilities to join and use Twitter. The details of the same are elaborated
in the following section.
3. PROPOSED TWITTER DATA EXTRACTION
FRAMEWORK (TDEF 1.0)
As discussed in section ii, Twitter has various accessibility features for normal
people [11],[12],[21],[23],[26],[27],[14],[18]. However, it is well realized that Twitter
should improve its accessibility among physically challenged prospective users [17].
In the past for many years, Twitter limited its use to traditional features for content
creation [13]. However, soon it began rolling out effective and engaging features to
allow more users to interact with real-time information in a more meaningful way [21].
In 2015, Twitter introduced many extensions to its timeline to leverage usage and
increase the number of active users [18]. However, after thorough timeline analysis
we realized that despite efforts, Twitter has as yet failed to introduce any timeline
enhancements that could bring people with disabilities easily on board. Though many
disabled have been absorbed by Twitter, many still hesitate [17].
Thus, we propose a real-time approach to include disabled people on Twitter.
Taking cue [15] and amplifying Twitter values for the people, our research majorly
focussed on the accessibility of Voice, Video, Images and Text to support people with
different challenges and allow them to express and present their opinions without any
technical barriers. We now briefly explain our approach for different kinds of tweets:
3.1. VOICE TWEET
After Twitter’s recent experiment of Voice tweets for the iOS app [11], users could
tweet their voice using the voice icon. Users could initially record up to 140 seconds of
audio. If the audio message was longer than 140 seconds, it will be automatically
threaded up to 25 audio tweets. We realized that the feature could be made more
inclusive for the disabled community through transcription and translation.
Through transcription, the user could read the text from the transcribed audio.
Through automated translation captions, users could also read the text in different
languages which would make audio tweets accessible to the hearing-impaired
audience.
3.2. VIDEO TWEET
To make video tweets accessible, we performed operations to generate a vocal
summary of video tweets with speech and without speech. Summarized text will help
the users to engage with content in a short period in more customized ways.
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
3.3. IMAGE/GIF TWEET
When users tweet images there is an additional feature to compose descriptions to
make the photos more accessible to people with disabilities including people who are
blind and have low vision.
In the alt-text feature for photos, add a short description under the limitation of 1000
characters for the audience to understand the type of content and increase
engagement in a personalized way. If posted, the description of the image won't be
visible and edited but visually impaired people will have access to the alt-text
description through screen readers [12].
To increase access to images globally, we developed the feature of vocal summary
for the image on the timeline which will help the audience with disabilities to listen to
the summary of activity performed in the images. This image summary will support
people understand the background without depending on the audience to let them
add descriptions in the alt-text feature.
3.4. TEXT TWEET
Traditional text tweeting was the historical feature launched by Twitter, the micro-
blogging platform in 2006 [13]. The product emerged with the limitation of text tweets
up to 140 characters which allowed the potential users to compose the content on the
social media platform and interact with relevant audiences.
In 2017, Twitter doubled its character limit from 140 to 280 for the audience to
share their real-time information in a more meaningful way with the support of 40
languages except for Japanese, Chinese and Korean (JCK) [18].
Using the 280-character tweet feature should not be limited to the audience without
any physical challenges. Consider the audience globally with different languages and
different disabilities. We thus developed Vocal Reading to let users listen to the tweet
they wish for. This will enhance the accessibility creatively without focusing to see on
the device. The feature shall also save the eyes from longer screen time.
4. IMPLEMENTATION DETAILS
Twitter has played a vital role in the field of content creation platforms and
significantly attracted millions of users in the past two decades all over the world [9].
The world has witnessed the power of Twitter where leaders to teenagers have
actively participated with tweets and led the reach of tweets to the trending section
[15],[19],[20].
In the following section, we will propose a model where the visibility of the tweets
will have a high reach to the potential audience which also includes people with
disabilities like visually impaired, hearing impaired, etc.
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.44 | Iss.12 | N.2 April - June 2023
37
Our model targeted videos, audio, text, photos, and gifs where we fetched real-time
tweets from Twitter and stored them in the database using .CSV files containing text,
media and URLs. Figure 2 below depicts a sample of the same.
We trained our model on the real-time tweets where we scrapped data in
November 2020 and stored 1000 tweets which majorly included tweets including text,
URL and media attachments.
4.1. EXTRACTION OF TWEETS
Real-time Tweets were extracted from Twitter using Twitter API (Application
Programming Interface) to analyze and learn behavior, direct interaction, locations
and other significant resources [28].
To analyze whether the multimedia and non-multimedia content will have more
access to people with disability, we sampled 1000 real-time tweets using our 3-way
technique described below in Figure 2.
As seen in Figure 2., Twitter API was used to generate authenticate credentials
including keys and tokens. After successfully generating Twitter API V2 endpoints,
real-time tweets are fetched to examine in the further model and stored in .CSV files.
The tweets were filtered based on their content type and data was cleaned of non-
relevant information like emoji, incomplete links, unwanted symbols, extra spaces and
retweets. Once cleaned, the significant labeled non-media information was stored in
respective CSV files and media files were maintained separately.
Figure 2. The 3-way technique for Data extraction using Twitter API
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.44 | Iss.12 | N.2 April - June 2023
38
Our model targeted videos, audio, text, photos, and gifs where we fetched real-time
tweets from Twitter and stored them in the database using .CSV files containing text,
media and URLs. Figure 2 below depicts a sample of the same.
We trained our model on the real-time tweets where we scrapped data in
November 2020 and stored 1000 tweets which majorly included tweets including text,
URL and media attachments.
4.1. EXTRACTION OF TWEETS
Real-time Tweets were extracted from Twitter using Twitter API (Application
Programming Interface) to analyze and learn behavior, direct interaction, locations
and other significant resources [28].
To analyze whether the multimedia and non-multimedia content will have more
access to people with disability, we sampled 1000 real-time tweets using our 3-way
technique described below in Figure 2.
As seen in Figure 2., Twitter API was used to generate authenticate credentials
including keys and tokens. After successfully generating Twitter API V2 endpoints,
real-time tweets are fetched to examine in the further model and stored in .CSV files.
The tweets were filtered based on their content type and data was cleaned of non-
relevant information like emoji, incomplete links, unwanted symbols, extra spaces and
retweets. Once cleaned, the significant labeled non-media information was stored in
respective CSV files and media files were maintained separately.
Figure 2. The 3-way technique for Data extraction using Twitter API
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
Table 2. The Statistic of extracted real-time Tweets – November 2020
Table 2 above lists the statistic to better understand the behavior of extracted
tweets, we have filtered the random sample of 1000 tweets from November 2020
according to the type of content where 400 tweets are text without any media
attachments and 600 tweets are text with embedded multimedia where 30 tweets are
audios by iOS users only, 230 tweets are videos with speech and without speech, 180
tweets are images, 45 tweets are GIF, 115 tweets are text with clickable URL.
4.2. TRANSCRIPTION
To explore the extent of Google services and API (Application Programming
Interface), the most popular Google Cloud Speech-to-text API was used to convert
Audio tweets and video tweets with speech into text which will be output as captions
and subtitles respectively.
Separating tweets after data cleaning summed up to 260 tweets in total where 30-
audio tweets and 230 video tweets were taken into consideration. The URL of the
audio and video (with speech) were imported from the CSV files and after researching
the frequency of media type, 5 different categories were observed to see which type
of content significantly added value to reach visually impaired audiences.
Table 3. Frequency of different categories of video -230 Tweets
We examined different video contents where out of 230 tweets, Table 3 describes
the frequency of occurrence of different tweets where 55.5 % of the potential audience
Tweet Type Number of tweets
Texts (iOS and Android) 400
Audios (iOS only) 30
Videos (iOS and Android) 230
Images (iOS and Android) 180
GIF (iOS and Android) 45
Text with URL (iOS and Android) 115
Categorical Video content Frequency
Videos 55.5 %
Speechless Video 7.8 %
Short Video Tweets 9.3 %
Advertisements 25.2 %
Videos with text/quotes 2.2 %
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.44 | Iss.12 | N.2 April - June 2023
39
generally tweeted videos with an average length of 1 minute, 7.8 % of videos have no
speech, 9.3% videos are short with an average length less than 1 minute,25.2%
videos are advertisement which is promoted by social brands and companies for
global users, 2.2% are videos which are either quotes or universal facts/texts.
Considering video and audio tweets, noise filtration was performed to remove the
unwanted audio and differentiate the original voice of the person.
Using the Google Speech-to-text API, the audio and videos were transcribed into
text and displayed in the form of captions and subtitles to the user. This technique is
thus proposed to increase the reach to hearing-impaired people where earlier the
videos were less accessible to people with hearing and vision problems. Our code
also facilitates Google translation with the support of almost 108 languages that can
translate the captioned transcribed text into their native language.
4.3. IMAGE CAPTIONING
Image captioning is one of the most popular features in the open-source
communities and Industry [9]
. Caption generation is an implementation of Deep
Learning in which the model is processed and generates captions or short
descriptions concerning the trained model. There is a huge probability that the model
will not return accurate captions or descriptions and for better results, a huge amount
of labelled dataset is required.
Separating 180 real-time image tweets which were stored in CSV files in the form
of tweet URLs, our model was prepared using open-source tools to caption different
categorical images.
The model aimed to not depend on the author of the original tweet containing
media contents to write a short description under a character size of 1000 in the ALT-
TEXT feature. The image captioning model is an independent feature that will facilitate
over millions of active users and remove the barrier of limited access and networking
opportunities.
To examine which part of the photo is detected to generate the caption, an
Attention-deep learning model which is similar to the ‘Show, Attend and Tell’ paper
was used [1],[30]. The model was trained using the MS-COCO dataset [6] with 40,000
images.
The deep learning InceptionV3 model was used as the feature extractor and then
an encoder-decoder model was trained for the generation of desired captions/
descriptions for new images.
4.4. TEXT SUMMARIZATION
Twitter is experimenting with media tweets containing audio and video for auto-
captioning. Considering the testing feature of Audio tweets by Twitter for iOS users
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.44 | Iss.12 | N.2 April - June 2023
40
generally tweeted videos with an average length of 1 minute, 7.8 % of videos have no
speech, 9.3% videos are short with an average length less than 1 minute,25.2%
videos are advertisement which is promoted by social brands and companies for
global users, 2.2% are videos which are either quotes or universal facts/texts.
Considering video and audio tweets, noise filtration was performed to remove the
unwanted audio and differentiate the original voice of the person.
Using the Google Speech-to-text API, the audio and videos were transcribed into
text and displayed in the form of captions and subtitles to the user. This technique is
thus proposed to increase the reach to hearing-impaired people where earlier the
videos were less accessible to people with hearing and vision problems. Our code
also facilitates Google translation with the support of almost 108 languages that can
translate the captioned transcribed text into their native language.
4.3. IMAGE CAPTIONING
Image captioning is one of the most popular features in the open-source
communities and Industry [9]. Caption generation is an implementation of Deep
Learning in which the model is processed and generates captions or short
descriptions concerning the trained model. There is a huge probability that the model
will not return accurate captions or descriptions and for better results, a huge amount
of labelled dataset is required.
Separating 180 real-time image tweets which were stored in CSV files in the form
of tweet URLs, our model was prepared using open-source tools to caption different
categorical images.
The model aimed to not depend on the author of the original tweet containing
media contents to write a short description under a character size of 1000 in the ALT-
TEXT feature. The image captioning model is an independent feature that will facilitate
over millions of active users and remove the barrier of limited access and networking
opportunities.
To examine which part of the photo is detected to generate the caption, an
Attention-deep learning model which is similar to the ‘Show, Attend and Tell’ paper
was used [1],[30]. The model was trained using the MS-COCO dataset [6] with 40,000
images.
The deep learning InceptionV3 model was used as the feature extractor and then
an encoder-decoder model was trained for the generation of desired captions/
descriptions for new images.
4.4. TEXT SUMMARIZATION
Twitter is experimenting with media tweets containing audio and video for auto-
captioning. Considering the testing feature of Audio tweets by Twitter for iOS users
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
and Video tweets for iOS and Android, our Text Summarization model is designed in
support of the above Transcription model where audio and videos from tweets will be
summarized into different language short sentences.
Open-source Python library, pysummarization is used that implements Encoder/
Decoder based on LSTM (Long Short-Term Memory) for improving the accuracy of
the desired summarization by Sequence-to-Sequence (Seq2Seq) learning.
Our proposed model is coded in a 3-step format which considered 260 tweets from
CSV files.
1. The script reads the generated video/audio tweet transcript.
2. Summarize in not more than three sentences.
3. Display the summary to the audience.
The Text summarization model in support of the transcription model will work
together to deliver the summary of the transcribed video/audio tweets. This will
encourage active users to understand the content in the form of a summary and lead
to meaningful interaction with other users.
5. RESULTS
The results obtained from the above experiment were motivating enough to infer
that our model can significantly make an impact to fill the gap of content accessibility
for people with disability.
The experiment has also removed the linguistic barrier by making the social media
platform available to all in their specific native language.
As the current framework of Twitter supports features to the potential audience
globally with an average of 6000 tweets per second which is approximately 200 billion
tweets per year.
Further, our model is simple to operate as it can be implemented with a simple
interface of only 2 buttons, namely: speaker icon (Read Out Loud) and information
icon (transcript) which will help the disabled user to listen to the text tweets and read
out the transcripts of multi-media contents like audio, video, image captioning and
summaries. A simulation of the same is depicted in Fig. 3, 4 and 5 below.
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.44 | Iss.12 | N.2 April - June 2023
41
Figure 3. Transcription of Audio Tweet using Information Button (‘i’)
Figure 4. Transcription of Video Tweet followed by Summarization using the ‘i’ button.
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.44 | Iss.12 | N.2 April - June 2023
42
Figure 3. Transcription of Audio Tweet using Information Button (‘i’)
Figure 4. Transcription of Video Tweet followed by Summarization using the ‘i’ button.
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
Figure 5. Read Out Loud Image caption using the speaker button
The above figures simulate our proposed framework for better accessibility of user
timelines on Twitter. Two major icons knowingly as the speaker and information button
support the proposed model which will leverage the open-source community along
with communities for disabled people.
In Figure 2. the iconic information button will generate the transcript of the audio
tweet and make it accessible majorly for hearing impaired people and other potential
audiences. In Figure 3, a video tweet is transcribed through the information button
followed up by a summary of the user’s primary language. The transcribed text
generated in Figure 4. after image captioning can be read out loud through the iconic
speaker button which is also compatible in Figure 2 and Figure 3 respectively.
However, after obtaining results from our proposed model which implements
Natural Language Processing, Transcription, Translation, and Summarization our
approach of using the open-source library for transcript summarization may not yield
better performance but there is always room for improvement and global contribution
in the community.
6. CONCLUSION
In this research, we analyzed different types of disabled people who need access
to Twitter with the advancement of technology and compared the existing features in
the current framework of user’s timeline with our proposed model which is specialized
to fill the gap for disabled people community and leverage the value of inclusion and
diversity.
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.44 | Iss.12 | N.2 April - June 2023
43
The proposed model we coined as TDEF1.0, allows immediate access to the
content and works independently without any indulgence of other people.
Concerning the current usage of Twitter by millions of active users, we have
examined the pattern of tweets from our dataset of 1000 tweets that infer that people
can improve their content creation style while keeping accessibility criteria in their
mind.
Our study also suggests that people should be aware of some DOs and DON’T
while creating and uploading content on social media platforms.
1. Active users should use camel case during Hashtags so that the screen reader
can read the words separately like #EasyToRead.
2. Avoid using different fonts for a tweet as the screen reader will mess with the
font name and actual word during screen reading.
3. Try to avoid using unnecessary emoji, special characters, abbreviations, GIFS,
extra spaces, etc.
4. Always add an Alt-Text short description if any image is uploaded on Twitter.
7. FUTURE SCOPE
After researching Twitter for everyone, a possibility arises to make Twitter better
every day with the advancement of technology and open-source tools. Our goal is to
bring diversity and inclusion to the micro-blogging platform and uplift the power of
disabled people.
In the future, we would want to bring focus on maintaining a dashboard for insights
and statistics on the total number of accounts registered for people with disabilities to
track their activities like the number of multi-media/ non-media tweets, retweets,
replies, comments, likes in a year to generate overall usage of the product.
Twitter is filled with millions of bot accounts and a bot account can be
advantageous to expand our support to the disabled people community by creating a
support bot that will exclusively work during accessibility issues and prepares a report
of the technical issues faced by the users on time.
Our research is supported by open-source technologies and considering the
perspective of different audiences, more advancement and modifications can be done
to make Twitter for everyone possible.
REFERENCES
(1) Allahyari, M., Pouriyeh, S., Assefi, M., Safaei, S., D., E., B., J., & Kochut, K.
(2017). Text Summarization Techniques: A Brief Survey. International Journal of
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.44 | Iss.12 | N.2 April - June 2023
44
The proposed model we coined as TDEF1.0, allows immediate access to the
content and works independently without any indulgence of other people.
Concerning the current usage of Twitter by millions of active users, we have
examined the pattern of tweets from our dataset of 1000 tweets that infer that people
can improve their content creation style while keeping accessibility criteria in their
mind.
Our study also suggests that people should be aware of some DOs and DON’T
while creating and uploading content on social media platforms.
1. Active users should use camel case during Hashtags so that the screen reader
can read the words separately like #EasyToRead.
2. Avoid using different fonts for a tweet as the screen reader will mess with the
font name and actual word during screen reading.
3. Try to avoid using unnecessary emoji, special characters, abbreviations, GIFS,
extra spaces, etc.
4. Always add an Alt-Text short description if any image is uploaded on Twitter.
7. FUTURE SCOPE
After researching Twitter for everyone, a possibility arises to make Twitter better
every day with the advancement of technology and open-source tools. Our goal is to
bring diversity and inclusion to the micro-blogging platform and uplift the power of
disabled people.
In the future, we would want to bring focus on maintaining a dashboard for insights
and statistics on the total number of accounts registered for people with disabilities to
track their activities like the number of multi-media/ non-media tweets, retweets,
replies, comments, likes in a year to generate overall usage of the product.
Twitter is filled with millions of bot accounts and a bot account can be
advantageous to expand our support to the disabled people community by creating a
support bot that will exclusively work during accessibility issues and prepares a report
of the technical issues faced by the users on time.
Our research is supported by open-source technologies and considering the
perspective of different audiences, more advancement and modifications can be done
to make Twitter for everyone possible.
REFERENCES
(1) Allahyari, M., Pouriyeh, S., Assefi, M., Safaei, S., D., E., B., J., & Kochut, K.
(2017). Text Summarization Techniques: A Brief Survey. International Journal of
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
Advanced Computer Science and Applications, 8(10). https://doi.org/10.14569/
ijacsa.2017.081052
(2) Ayo, F. E., Folorunso, O., Ibharalu, F. T., & Osinuga, I. A. (2020). Machine
learning techniques for hate speech classification of twitter data: State-of-The-
Art, future challenges and research directions. In Computer Science Review
(Vol. 38, p. 100311). Elsevier Ireland Ltd. https://doi.org/10.1016/
j.cosrev.2020.100311
(3) Boyd, D. M., & Ellison, N. B. (2007). Social network sites: Definition, history, and
scholarship. Journal of Computer-Mediated Communication, 13(1), 210–230.
https://doi.org/10.1111/j.1083-6101.2007.00393.x
(4) Chatziadam, P., Dimitriadis, A., Gikas, S., Logothetis, I., Michalodimitrakis, M.,
Neratzoulakis, M., Papadakis, A., Kontoulis, V., Siganos, N., Theodoropoulos,
D., Vougioukalos, G., Hatzakis, I., Gerakis, G., Papadakis, N., & Kondylakis, H.
(2020). TwiFly: A data analysis framework for Twitter. Information (Switzerland),
11(5), 1–14. https://doi.org/10.3390/INFO11050247
(5) Chiarella, D., Yarbrough, J., & Jackson, C. A. L. (2020). Using alt text to make
science Twitter more accessible for people with visual impairments. Nature
Communications, 11(1), 10–12. https://doi.org/10.1038/s41467-020-19640-w
(6) COCO Dataset | Papers With Code. (n.d.). Retrieved February 18, 2023, from
https://paperswithcode.com/dataset/coco
(7) Desselle, S. P. (2017). The use of Twitter to facilitate engagement and reflection
in a constructionist learning environment. Currents in Pharmacy Teaching and
Learning, 9(2), 185–194. https://doi.org/10.1016/j.cptl.2016.11.016
(8) Ellis, K., & Kent, M. (2010). Community Accessibility : Tweeters Take
Responsibility for an Accessible Web 2.0. Fast Capitalism, 7(1), 39–46.
(9) Gleason, C., Carrington, P., Cassidy, C., Morris, M. R., Kitani, K. M., & Bigham,
J. P. (n.d.). “ It ’ s almost like they ’ re trying to hide it ": How User-Provided
Image Descriptions Have Failed to Make Twitter Accessible. 549–559. https://
doi.org/10.1145/3308558.3313605
(10) Hellemans, J., Willems, K., & Brengman, M. (2020). Daily Active Users of Social
Network Sites: Facebook, Twitter, and Instagram-Use Compared to General
Social Network Site Use (pp. 194–202). Springer, Cham. https://doi.org/
10.1007/978-3-030-47595-6_24
(11) How to Direct Message (DM) on Twitter | Twitter Help. (n.d.). Retrieved May 13,
2021, from https://help.twitter.com/en/using-twitter/direct-messages
(12) How to make images accessible for people. (n.d.). Retrieved May 13, 2021, from
https://help.twitter.com/en/using-twitter/picture-descriptions
(13) jack on Twitter: “just setting up my twttr” / Twitter. (n.d.). Retrieved May 14, 2021,
from https://twitter.com/jack/status/20
(14) Khan, I., Naqvi, S. K., Alam, M., & Rizvi, S. N. A. (2017). An efficient framework
for real-time tweet classification. International Journal of Information Technology
(Singapore), 9(2), 215–221.
(15) Khatua, A., Khatua, A., & Cambria, E. (2019). A tale of two epidemics:
Contextual Word2Vec for classifying twitter streams during outbreaks.
Information Processing and Management, 56(1), 247–257. https://doi.org/
10.1016/j.ipm.2018.10.010
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.44 | Iss.12 | N.2 April - June 2023
45
(16) Little-known Twitter feature can help blind and visually impaired people use the
site | The Independent | The Independent. (n.d.). Retrieved May 13, 2021, from
https://www.independent.co.uk/life-style/gadgets-and-tech/news/twitter-blind-
users-alt-text-accessibility-features-ios-android-app-website-compose-image-
descriptions-a8143391.html
(17) Morris, M. R., Zolyomi, A., Yao, C., Bahram, S., Bigham, J. P., & Kane, S. K.
(2016). “With most of it being pictures now, I rarely use it”: Understanding
Twitter’s evolving accessibility to blind users. Conference on Human Factors in
Computing Systems - Proceedings, 5506–5516. https://doi.org/
10.1145/2858036.2858116
(18) Sujay, R., Pujari, J., Bhat, V. S., & Dixit, A. (2018). Timeline Analysis of Twitter
User. Procedia Computer Science, 132, 157–166. https://doi.org/10.1016/
j.procs.2018.05.179
(19) Talbot, C. V, Dwyer, S. T. O., Clare, L., & Heaton, J. (2021). The use of Twitter by
people with young-onset dementia : A qualitative analysis of narratives and
identity formation in the age of social media. https://doi.org/
10.1177/14713012211002410
(20) Tang, Y., & Hew, K. F. (2017). Using Twitter for education: Beneficial or simply a
waste of time? Computers and Education, 106, 97–118. https://doi.org/10.1016/
j.compedu.2016.12.004
(21) The Top Twitter Updates You Need to Know: January 2021. (n.d.). Retrieved
May 13, 2021, from https://blog.hootsuite.com/twitter-updates/
(22) Twitter apologies for not making voice tweets accessible to deaf people | The
Independent | The Independent. (n.d.). Retrieved May 13, 2021, from https://
www.independent.co.uk/life-style/gadgets-and-tech/news/twitter-voice-notes-
deaf-accessible-ios-iphone-app-update-a9574966.html
(23) Twitter Fleets Introduced to Encourage More Tweets. (n.d.). Retrieved May 13,
2021, from https://www.searchenginejournal.com/twitter-fleets/387927/
(24) Twitter invited a deaf person to test out Spaces – here’s what they think… –
Liam O’Dell. (n.d.). Retrieved May 13, 2021, from https://liamodell.com/
2021/01/16/twitter-spaces-deaf-captions-clubhouse-audio/
(25) Twitter just rolled out a feature that’s inaccessible to disabled users. (n.d.).
Retrieved May 13, 2021, from https://slate.com/technology/2020/06/twitter-voice-
tweets-accessibility.html
(26) Twitter launches Bookmarks – an anonymous way to save tweets - The Courier.
(n.d.). Retrieved May 13, 2021.
(27) Twitter voice Tweets – how to Tweet with your voice or audio. (n.d.). Retrieved
May 13, 2021, from https://help.twitter.com/en/using-twitter/voice-tweet
(28) Ugochi, O., Prasad, R., Odu, N., Ogidiaka, E., & Ibrahim, B. H. (2022). Customer
opinion mining in electricity distribution company using twitter topic modeling and
logistic regression. International Journal of Information Technology (Singapore),
14(4), 2005–2012.
(29)
Valencia, S., Kirabo, L., Carter, E. J., Bennett, C. L., & Pavel, A. (2020). Disability
and the COVID-19 Pandemic Using Twitter to Understand Accessibility during
Rapid Societal Transition. i. https://doi.org/10.1145/3373625.3417023
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.44 | Iss.12 | N.2 April - June 2023
46
(16) Little-known Twitter feature can help blind and visually impaired people use the
site | The Independent | The Independent. (n.d.). Retrieved May 13, 2021, from
https://www.independent.co.uk/life-style/gadgets-and-tech/news/twitter-blind-
users-alt-text-accessibility-features-ios-android-app-website-compose-image-
descriptions-a8143391.html
(17) Morris, M. R., Zolyomi, A., Yao, C., Bahram, S., Bigham, J. P., & Kane, S. K.
(2016). “With most of it being pictures now, I rarely use it”: Understanding
Twitter’s evolving accessibility to blind users. Conference on Human Factors in
Computing Systems - Proceedings, 5506–5516. https://doi.org/
10.1145/2858036.2858116
(18) Sujay, R., Pujari, J., Bhat, V. S., & Dixit, A. (2018). Timeline Analysis of Twitter
User. Procedia Computer Science, 132, 157–166. https://doi.org/10.1016/
j.procs.2018.05.179
(19) Talbot, C. V, Dwyer, S. T. O., Clare, L., & Heaton, J. (2021). The use of Twitter by
people with young-onset dementia : A qualitative analysis of narratives and
identity formation in the age of social media. https://doi.org/
10.1177/14713012211002410
(20) Tang, Y., & Hew, K. F. (2017). Using Twitter for education: Beneficial or simply a
waste of time? Computers and Education, 106, 97–118. https://doi.org/10.1016/
j.compedu.2016.12.004
(21) The Top Twitter Updates You Need to Know: January 2021. (n.d.). Retrieved
May 13, 2021, from https://blog.hootsuite.com/twitter-updates/
(22) Twitter apologies for not making voice tweets accessible to deaf people | The
Independent | The Independent. (n.d.). Retrieved May 13, 2021, from https://
www.independent.co.uk/life-style/gadgets-and-tech/news/twitter-voice-notes-
deaf-accessible-ios-iphone-app-update-a9574966.html
(23) Twitter Fleets Introduced to Encourage More Tweets. (n.d.). Retrieved May 13,
2021, from https://www.searchenginejournal.com/twitter-fleets/387927/
(24) Twitter invited a deaf person to test out Spaces – here’s what they think… –
Liam O’Dell. (n.d.). Retrieved May 13, 2021, from https://liamodell.com/
2021/01/16/twitter-spaces-deaf-captions-clubhouse-audio/
(25) Twitter just rolled out a feature that’s inaccessible to disabled users. (n.d.).
Retrieved May 13, 2021, from https://slate.com/technology/2020/06/twitter-voice-
tweets-accessibility.html
(26) Twitter launches Bookmarks – an anonymous way to save tweets - The Courier.
(n.d.). Retrieved May 13, 2021.
(27) Twitter voice Tweets – how to Tweet with your voice or audio. (n.d.). Retrieved
May 13, 2021, from https://help.twitter.com/en/using-twitter/voice-tweet
(28) Ugochi, O., Prasad, R., Odu, N., Ogidiaka, E., & Ibrahim, B. H. (2022). Customer
opinion mining in electricity distribution company using twitter topic modeling and
logistic regression. International Journal of Information Technology (Singapore),
14(4), 2005–2012.
(29) Valencia, S., Kirabo, L., Carter, E. J., Bennett, C. L., & Pavel, A. (2020). Disability
and the COVID-19 Pandemic Using Twitter to Understand Accessibility during
Rapid Societal Transition. i. https://doi.org/10.1145/3373625.3417023
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
(30) Yadav, A. K., Singh, A., Dhiman, M., Vineet, Kaundal, R., Verma, A., & Yadav, D.
(2022). Extractive text summarization using deep learning approach.
International Journal of Information Technology (Singapore), 14(5), 2407–2415.
ABOUT THE AUTHORS
Ms. Namrata Aggarwal
Ms. Namrata Aggarwal is currently working as an Analyst at KPMG and post-
graduated with a Master of Computer Application (M.C.A) from Bharati Vidyapeeth's
Institute of Computer Applications and Management (BVICAM) in June 2022.
Passionate about learning new things every day for career and life. She is an
acclaimed researcher in Artificial Intelligence who has also won #WomenHackAI,
5XHackathons and also been a Gold Microsoft Learn Student Ambassador.
Dr. Ritika Wason
Dr. Ritika Wason is working as Associate Professor with Bharati Vidyapeeth's
Institute of Computer Applications and Management (BVICAM), New Delhi. She is
also the managing editor for the International Journal of Information Technology (IJIT),
an official Journal of Bharati Vidyapeeth’s Institute of Computer Applications and
Management (BVICAM) co-published with Springer and UGC-Care Indexed and
Scopus indexed. An avid researcher, she is also the editor for CSI Communications, a
monthly magazine published by the Computer Society of India (CSI). A certified
mendeley trainer she has trained several professionals and scholars on mendeley. A
researcher she has also authored many books and papers published by many leading
publishers.
Dr. Parul Arora
Dr. Parul Arora is working as Associate Professor with Bharati Vidyapeeth's
Institute of Computer Applications and Management (BVICAM), New Delhi. An avid
researcher she has many research papers published in many renowned journals and
conferences.
Ms. Aruna Tomar
Ms Aruna Tomar is working as an Assistant Professor with Bharati Vidyapeeth's
Institute of Computer Applications and Management (BVICAM), New Delhi. An avid
researcher she has many research papers published in many renowned journals and
conferences.
Mr. Devansh Arora
Mr. Devansh Arora is a student at Indraprastha Institute of Information Technology
(IIIT),
Delhi. An artificial intelligence and machine learning enthusiast he has many
projects and papers to his credit.
https://doi.org/10.17993/3ctecno.2023.v12n2e44.31-47
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254-4143
Ed.44 | Iss.12 | N.2 April - June 2023
47
