Requests for Pidgin 3

Pidgin
1

I’ve been a long-time user of Pidgin2, but it does show its age and I have found myself having to open native/web clients more often than I would like to, or write python scripts to fix issues.

Below is a non-exhaustive list of suggestions and requests that I would like to see in Pidgin3. I understand that some of them will be up to individual plugins, but still the core Pidgin needs to provide the infrastructure to make them possible.

Message store

One of the big advantages of Pidgin over the native clients is the ability to keep local history outside of the server’s control. Pidgin2’s handling of history is limited (especially if written to HTML files), but I assume Pidgin3 will use a proper DB.

* Many servers allow rewriting of history by editing or deleting messages. I would like Pidgin to support this functionality but implement it using versioning, so that previous versions of a message are preserved in the DB and are easily accessible by the user.

* The message store should allow downloading full or partial history from the server, while still supporting versioning even if the server does not include a modification timestamp. This can be done by keeping a message hash as a part of the message record. The actual synchronization is protocol-dependent and will likely be handled by plugins.

* Some protocols support message attachments. I’d like Pidgin to allow storing those attachments locally (either in the DB or file system) or as URLs pointing to the server-side artifacts (user choice per connection). having it support versioning as well would be best.

* The DB schema should allow for merging and/or synchronizing between different instances. For example, I use Pidgin both on my desktop and my laptop (when travelling) and would like the ability to have the same history on both, even if a manual step is involved. I don’t mind writing SQL scripts if the schema is documented.

User interface

* Pidgin2 supports a 2-level hierarchy: groups and contacts/room. I’d like Pidgin3 to support a multi-level tree for more flexible organization. It is sorely needed for organizing channels of Discord servers.

* Support custom badges for messages, to indicate various statuses (for example: edited/deleted message) and custom context menus (invocable from the badge or the message). Operations can include: displaying previous versions of edited/deleted messages, quoting the message, reacting to the message, or other operations registered by plugins.

* Similarly allow plugins to register operations on context menus of conversations, hotkeys in the input field, server-specific “slash commands”, etc.

* Better formatting support. Currently formatting is an all or nothing proposition. Some protocols support subsets or supersets of the formatting options.

* Better emoji support. Themes are great, but almost all platforms support searching in the emoji input popups.

More to come…

Thank you!

2

One of the big advantages of Pidgin over the native clients is the ability to keep local history outside of the server’s control. Pidgin2’s handling of history is limited (especially if written to HTML files), but I assume Pidgin3 will use a proper DB.

Already planned and mostly implemented. It has a few outstanding issues that are blocked by other things at the moment. Right now only sqlite is supported, but the backend can be extended with plugins.

The message store should allow downloading full or partial history from the server, while still supporting versioning even if the server does not include a modification timestamp. This can be done by keeping a message hash as a part of the message record. The actual synchronization is protocol-dependent and will likely be handled by plugins.

We’ve added a command line tool for importing/exporting history from pidgin 2 or whatever and this is something that could go in there, but would probably need to be in a user interface because it’d require a logged in account.

Some protocols support message attachments. I’d like Pidgin to allow storing those attachments locally (either in the DB or file system) or as URLs pointing to the server-side artifacts (user choice per connection). having it support versioning as well would be best.

Image attachments will be cached locally as they’re part of the message. File attachments are problematic because someone could attach a huge file if it’s automatically downloaded it could exhaust your free disk space which is something we obviously don’t want to do. We could set a limit for that, but we’d have to make it user configurable which is going to lead to problems regardless. That said, there might be something here.

When it comes to URL previews, the intent is to support both open graph and summary tiles, but there’s a lot of privacy issues there that need to be solved first.

Regardless, records of all attachments including ones that are downloaded will be stored in the database with their relation to the message they are from. The raw data will be in a disk cache that we need to finish building as it is currently blocking an overhaul of cleaning up the avatar api that’s replacing the old buddy icon api.

The DB schema should allow for merging and/or synchronizing between different instances. For example, I use Pidgin both on my desktop and my laptop (when travelling) and would like the ability to have the same history on both, even if a manual step is involved. I don’t mind writing SQL scripts if the schema is documented.

As I mentioned earlier there’s already a cli built for this but needs additional work.

Pidgin2 supports a 2-level hierarchy: groups and contacts/room. I’d like Pidgin3 to support a multi-level tree for more flexible organization. It is sorely needed for organizing channels of Discord servers.

We’re removing the emphasis on the contact list by making it very searchable and filterable list. This will also be helped by the fact that conversation layout will be saved between when the program is open so you don’t need to find the person you talk to all the time in the contact list to open a window as it’ll already be open.

Support custom badges for messages, to indicate various statuses (for example: edited/deleted message) and custom context menus (invocable from the badge or the message). Operations can include: displaying previous versions of edited/deleted messages, quoting the message, reacting to the message, or other operations registered by plugins.

This is all already in progress and in many different states.

Similarly allow plugins to register operations on context menus of conversations, hotkeys in the input field, server-specific “slash commands”, etc.

Plugins have always been able to do all of this.

Better formatting support. Currently formatting is an all or nothing proposition. Some protocols support subsets or supersets of the formatting options.

This is tricky to get right, but I think we’re close. Can you provide examples where things aren’t supported?

Better emoji support. Themes are great, but almost all platforms support searching in the emoji input popups.

With Slack, Discord, and Twitch emotes are a huge problem. We have a number of ideas on how to address them but we have a lot of stuff in the pipe line before we get to them.

Also when it comes to the current state of Pidgin 3, you will probably want to check out my State of the Bird addresses as much of this stuff has already been covered there.

3

But, please, allow storing messages in plain text files (as today), not only in DB format. Pidging allows us to free messages from server’s control, and text format allows messages to be easily processed outside Piding control.

Searching for emoji - at least on Windows, pressing Windows-“.” opens searchable emoji window.

But other suggestion about emoji support. Maybe Pidgin could auto-create/auto-update emoji pictures, using current Unicode standard. As in my command line tool:

4

But, please, allow storing messages in plain text files (as today), not only in DB format. Pidging allows us to free messages from server’s control, and text format allows messages to be easily processed outside Piding control.

Dropping the database means we can properly order edits and stuff in place which is why pidgin2 can’t do anything like edits, redactions, replies, threads, reactions, etc. That said, messages can be exported via the cli tool.

Searching for emoji - at least on Windows, pressing Windows-“.” opens searchable emoji window.

Those emoji pickers typically don’t allow us to add custom emojis from stuff like Slack, Discord, Twitch, etc. So we have to build our own emoji picker that will allow us to do that.

But other suggestion about emoji support. Maybe Pidgin could auto-create/auto-update emoji pictures, using current Unicode standard. As in my command line tool:

Yes, standard emoji will be in the new picker automatically. “Smiley Themes” have already been removed from the Pidgin 3 code base.

5

I have some auto-scripts connected to (current - plain text) pidgin message store… Maybe allowing text-dump of messages, with current format (folders for each buddy/contact, file for every chat session) as parallel to DB (something like 'DB operation log file :slight_smile: ) could be added to Pidgin3?

How do you want to allow ‘forwarding’ of e.g. Discord message, with discord-specific emojis, to Slack, WhatsApp, etc.?
Main usage of Pidgin, for me, is one messenger UI for all channels (currently only facebook outside Pidgin, as facebook forced me to use two-factor authentication and current Pidgin plugin cannot handle this); and I’m frequently use forward (as copy&paste) between different services.

“standard”, means Unicode in which version?
My proposition is auto-updating images/icons for emojis, as standard is updated once per year.
Simply, take
http://www.unicode.org/emoji/charts/emoji-proposals.html#by_emoji
and get image from second column for sequence from first column.

6

I have some auto-scripts connected to (current - plain text) pidgin message store… Maybe allowing text-dump of messages, with current format (folders for each buddy/contact, file for every chat session) as parallel to DB (something like 'DB operation log file :slight_smile: ) could be added to Pidgin3?

As I’ve been saying there is a cli tool for purple3 which is meant query as well as import and export logs. Output format hasn’t really been settled yet but it really could be anything and we can make it pluggable too.

I think the better question here is would it make more sense to add support for what your scripts are doing into libpurple itself rather than depending on a serialized logs format that we can’t even easily insert arbitrary messages into without being able to keep it chronological.

How do you want to allow ‘forwarding’ of e.g. Discord message, with discord-specific emojis, to Slack, WhatsApp, etc.?
Main usage of Pidgin, for me, is one messenger UI for all channels (currently only facebook outside Pidgin, as facebook forced me to use two-factor authentication and current Pidgin plugin cannot handle this); and I’m frequently use forward (as copy&paste) between different services.

I mean a copy paste is what it is, but Slack, Discord and others have custom emoji and the only reasonable way to paste them into a service that doesn’t have them is to use their text representation.

“standard”, means Unicode in which version?
My proposition is auto-updating images/icons for emojis, as standard is updated once per year.
Simply, take
Emoji Proposals, v15.1
and get image from second column for sequence from first column.

Standard means we update it when a new release is out. But we need a custom picker that supports custom emojis everywhere. For example, Slack supports standard unicode emojis, 3000 custom emojis per instance (last I checked), as well as converting text based emoji (what pidgin 2 calls smileys) into standard emojis. We need an emoji picker that does all of that. But it also needs to support other protocols, be resource efficient, and easy for users to understand, which means it’s going to take some time to get it right.

7

Hi Gary, thanks for the reply!

SQLite is great, especially as it comes bundled with many OSes.

What about the versioning? Does the implementation allow for storing and viewing messages that were subsequently edited/deleted?

In response to the import/export tool: hopefully it supports versioning and handling duplicates.

But that is not exactly what I was referring to. I meant fetching (and reconciling) old messages from the server while Pidgin is running and the user is logged in.

The Discord plugin for Pidgin2 has rudimentary functionality for fetching old messages for a conversation/channel from a server, but due to the limitations of the platform it does not do that very well. Fetching old messages is useful when one joins a room/channel that already has a conversation going, or when migrating from native clients to Pidgin, but if versioning is supported (hopefully!) and the server only keeps one the last version with only the original timestamp, then storing hashes can be used for distinguishing between new/existing/modified messages, and possibly even marking messages as deleted if they are not on the server.

The actual behaviour (when to fetch, how many messages, etc.) can be handled by protocol plugins, and hopefully be user-settable. For example: in 1:1 conversations or low traffic rooms, I’d like to fetch everything that is missing, but for high-traffic channels – especially those I visit infrequently – I find the Discord way of fetching messages on scrolling up to be quite useful.

That depends on the protocol. Discord saves the images (and other artifacts) server-side and includes URLs in the message (for attachments, embeds, etc.)

Why not use the file system? It allows for easy local cleanup, and hopefully Pidgin3 will be able to fall back to a URL if the artifact has been removed.

Also, how are you going to support reactions on messages?

Searching is nice, but so is being able to see at a glance which of the buddies in a particular (user-defined) group are online, or which channels/rooms have had activity since last visited (in Pidgin2 I set the conversations to automatically open, but an indicator on the list is more flexible). I’d like to reiterate my request for a tree-style list, and having is searchable and filterable is a bonus!

Great!

Some protocols support underline, others don’t. Similarly with strikeout, subscript/superscript, etc. Ideally a protocol plugin would be able to specify which subset it supports and possibly even add additional formatting options (possibly bracketed by tags if a visual representation is not possible)

I am currently using the emoji pop-up with a custom theme based on joypixels7, but with almost 1900 entries, and no way of searching, it is a bit clunky. I did repurpose the “text replacement” plugin to replace common shortcuts with emojis, but that’s a bit of a hack.

Unfortunately, the version of GTK that is bundled with Pidgin2 on Windows does not support Unicode display very well, and almost everything shows as tofu. Bleh.

I will!

Text format can always be generated with either a plugin or external tool. If the DB schema is public, it’s easy to manipulate the content as you see fit. @grim, please document it (even if it’s a WIP).

… which works reasonably in almost all programs, except for Pidgin2. It will insert a B&W symbol (not the emoji) if no filtering is applied, and nothing if you try to filter. See images:

1
1920×400 35.7 KB

2
2920×400 41.5 KB

That’s a pity. My eyesight is not what it used to be, and the ability to use a different collection of images (joypixels7 currently) at different sizes is very helpful.

I suspect by either sending the actual image, or falling back to a text “:identifier:”

Just make it extendable, start with something usable, then add features.

8

I listened to the presentation and wrote some comments in the corresponding topic.

Thank you!

9

What about the versioning? Does the implementation allow for storing and viewing messages that were subsequently edited/deleted?

We don’t have this implemented yet but the plan is to store all of this.

But that is not exactly what I was referring to. I meant fetching (and reconciling) old messages from the server while Pidgin is running and the user is logged in.

This will be up to the protocol or maybe we’ll add an action somewhere, but the current I idea is to load it when the user scrolls back in the history for the chat.

That depends on the protocol. Discord saves the images (and other artifacts) server-side and includes URLs in the message (for attachments, embeds, etc.)

No I mean, we’re going to cache image data just like a web browser does. There’s no reason why we should be hitting servers again and again for images we already have. Obviously there will be some timeout here, but we need the raw data to display it anyways so it only makes sense to cache it on disk.

Why not use the file system? It allows for easy local cleanup, and hopefully Pidgin3 will be able to fall back to a URL if the artifact has been removed.

We’re using the filesystem for the disk cache… But there will be record of them in the database. That’s what I said before, hopefully it was more clear this time.

Also, how are you going to support reactions on messages?

That is the intent yes. But we have a ton of stuff to do before we can get to that point.

Searching is nice, but so is being able to see at a glance which of the buddies in a particular (user-defined) group are online, or which channels/rooms have had activity since last visited (in Pidgin2 I set the conversations to automatically open, but an indicator on the list is more flexible). I’d like to reiterate my request for a tree-style list, and having is searchable and filterable is a bonus!

This is already a flat list, we’re not changing it. As for who’s online and stuff, the intent is that the contact list will allow you custom filters and stuff which should fill that need.

Some protocols support underline, others don’t. Similarly with strikeout, subscript/superscript, etc. Ideally a protocol plugin would be able to specify which subset it supports and possibly even add additional formatting options (possibly bracketed by tags if a visual representation is not possible)

This is more or less low priority right now. We’ll get it eventually, but there’s much bigger things to worry about.

I am currently using the emoji pop-up with a custom theme based on joypixels7, but with almost 1900 entries, and no way of searching, it is a bit clunky. I did repurpose the “text replacement” plugin to replace common shortcuts with emojis, but that’s a bit of a hack.

Unfortunately, the version of GTK that is bundled with Pidgin2 on Windows does not support Unicode display very well, and almost everything shows as tofu. Bleh.

Yes the pidgin 2 gtk 2 situation on windows sucks, but fixing it requires considerable work that is better spent working on pidgin 3.

Text format can always be generated with either a plugin or external tool. If the DB schema is public, it’s easy to manipulate the content as you see fit. @grim, please document it (even if it’s a WIP).

As I keep saying, we’re providing a cli tool for importing and exporting. Also, Pidgin is an open source project so of course the schema is public, but even then a .schema will work in any sqlite database. Anyways the current schema which was a proof of concept is here HGKeeper.

… which works reasonably in almost all programs, except for Pidgin2. It will insert a B&W symbol (not the emoji) if no filtering is applied, and nothing if you try to filter. See images:

Yes, pidgin 2 sucks, we know, that’s why we’re trying to get pidgin 3.0 alpha 1 done so people can move to it…

That’s a pity. My eyesight is not what it used to be, and the ability to use a different collection of images (joypixels7 currently) at different sizes is very helpful.

Well if there’s a font for it, that’d be the way to go for pidgin 3 and it looks like there is https://joypixels.com/fonts/

Just make it extendable, start with something usable, then add features.

Emoji is currently one of our biggest outstanding problems. “Just make it extendable” is an insult to the amount of engineering we’ve already put into it.

10

Yes yes, a bit of a failure in communication. I know how such caches work, what I meant is that the browser cache is opaque, while I hoped that the Pidgin3 disk cache will be more user friendly, so one will be able to, for example, find artifacts even when Pidgin is not running.

And also, the ability to fall back to just displaying a clickable URL to the server-side if the user decides to clean some cache entries due to disk size concerns.

Hopefully I was more clear as well :slight_smile:

Many of the things that I mentioned are “nice to have”, and I am mentioning them just to put them on the radar, or to get a confirmation that they are already on the radar (which seems to be the case for most of them, thanks!)

The only one that concerns me is the flat list, for the reasons I outlined in the other thread so I won’t repeat them here.

I do appreciate your responses even when you disagree with my reasoning.

I am a Pidgin user, not a Pidgin developer. I use it for communication so I will be able to switch to Pidgin3 when it achieves parity with Pidgin2 on the protocols that I use, which would take a while.

If fixing the emoji situation on Windows is as easy as bundling a newer version of GTK+2, then I will be ecstatic if it happens, otherwise I’ll have to live with the situation. Incidentally, I found a newer version but wasn’t able to make it work (I’m not familiar with GTK). Details in another thread.

Thanks!

Only for Mac, iOS or Android :frowning:
I’m not hung up on JoyPixels, they just provide a complete set of emojis for free that I could resize and create a theme from. Currently using 24px size.

No disrespect intended and your effort is very much appreciated. I do apologize for my poor choice of words. What I meant (and failed to convey) is that maybe it will be possible to release something that “sort of works” while the complete implementation is being worked on.

11

Yes yes, a bit of a failure in communication. I know how such caches work, what I meant is that the browser cache is opaque, while I hoped that the Pidgin3 disk cache will be more user friendly, so one will be able to, for example, find artifacts even when Pidgin is not running.

Unlikely, we need to avoid name collisions just like web browsers so they will most likely be hashed based on their content or something. Again, this isn’t implemented at all yet.

Why would we show a link instead of re-download the asset as the user has clearly requested it again?

Many of the things that I mentioned are “nice to have”, and I am mentioning them just to put them on the radar, or to get a confirmation that they are already on the radar (which seems to be the case for most of them, thanks!)

The forums isn’t a radar anyone going to remember see/check later. Also this is all super premature as no one is actively using Pidgin 3 as a daily driver not even me.

If fixing the emoji situation on Windows is as easy as bundling a newer version of GTK+2, then I will be ecstatic if it happens, otherwise I’ll have to live with the situation. Incidentally, I found a newer version but wasn’t able to make it work (I’m not familiar with GTK). Details in another thread.

We are not doing this. Every time we touch the windows build something else breaks in new and exciting ways. I’m sorry emoji are broken, I’m sorry Pidgin 2 crashes on exit on windows, but we’ve tried to fix both of those numerous times and it was a gigantic waste of time because it ended up breaking other things along the way.

Only for Mac, iOS or Android :frowning:

From the android section:

Library: JoyPixels 6.6
File format: .ttf

This should be installable on windows as windows uses ttf fonts.

No disrespect intended and your effort is very much appreciated. I do apologize for my poor choice of words. What I meant (and failed to convey) is that maybe it will be possible to release something that “sort of works” while the complete implementation is being worked on.

That is the literal goal of Alpha 1… There’s no mention of emojis there because they are a huge problem, the os specific emoji pickers will have to do when the alpha is released.

12

Not that I have the impression you are ‘taking requests’ for the software but can I ask if any consideration has been given to the GUI of Pidgin3? Will it be a massive departure from Pidgin2 or can we expect more of the same (in a good way, I mean!).

One of the reasons I am still using Pidgin2 on windows (despite the issues*) is because I absolutely hate the ‘one giant window’ all messenger apps have become. It’s workable on something like telegram where you basically have all your people/groups/chats in what is effectively a list of tabs (and they let you ‘pin’ them to the top for the ones you really frequent!), but it is a nightmare on Discord where you can really only look at one chat or DM at a time because of the ‘server’ organization method. Someone mashed up IRC and teamspeak and for some reason kept the worst part of both.

13

You can get a rough idea of what we’re going for (so far) from the demo during The State of the Bird (minus my goof up at the beginning). Eventually there will be the ability to pin conversations on the left, but there will also be the ability to split stuff out into additional windows too, but we’re a ways away from that.

14

It sounds like we’re a ways away from Pidgin3 being the solution I was hoping for, thank you for your time, though!

15
  1. A more attractive GUI
  2. Native and complete Facebook messenger support
    2.5 FaceBook Messenger video/audio calls

Thanks