As a single individual I can only really have a limited affect on the future. In order to maximize usefulness I've taken to contain my worry about existential AGI risk and longtermism as to where I can have an impact. A thought experiment which has left me concerned is would a singularity/take off AI event still find humans valuable to keep existing? I have to limit the scope of this worry to something I could and am needed to affect. Others are working on AGI safety such that I don't have to address paper-clip/grey-goo thought experiments here.
What I haven't seen addressed nor debated, are what traits keep us uniquely human (such as random thought experiments?) enough that an AGI would want to let us exist, and with our free-will intact? Would we need to grow such traits? How much would civilization grow by more people embodying these traits?
Assuming there's a set of reasons an AI would agree not to destroy us, what would the answers to those reasons look like? Our most lauded traits are humanitarian traits and values. What would it take for more of humanity to adopt and embody humanitarian traits and values?
Considering numerous thought experiments, given the likely sophistication of an AGI the traits and values ought to support individual uniqueness and randomness, self-determination, expression, exploration and persist enough to translate to positive actions across humanity. These traits and values would both indirectly shape views, and directly create humanitarian actions: enact both non local and local cooperation, do unselfish acts, be kind, hold stewardship and service in high regard -- err on the side of altruism, and practice more platonic love, don't let selfish or evolutionary need be the principle driver of behavior, and correct our failings. (Regarding altruism, in practice, as it's impossible to live completely altruistically, seek to do so in moderation; engineer for maximal effectiveness and increase leverage which lowers learning curves to enable more good.) If we as humans are able to embody and put forth these values we would be more likely a unique good within the universe.
By stating this, maybe I've made this ultimately selfish for humanity's own ends, but maybe I've allowed us to justify more of humanity being better in a secular manner.
Can we say humanity embodies a poetic beauty, unique among the cosmos? I hope We'll do better such that the future will say yes.
