Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1980 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.8609. Min Act: -0.0000

Data Index: 1751865 (The Pile)

Max Activating Token Index: 48

Text #1

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.5012. Min Act: -0.0000

Data Index: 234815 (The Pile)

Max Activating Token Index: 110

Text #2

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.4356. Min Act: -0.0000

Data Index: 273381 (The Pile)

Max Activating Token Index: 946

Text #3

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.3913. Min Act: -0.0000

Data Index: 57902 (The Pile)

Max Activating Token Index: 43

Text #4

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.2857. Min Act: -0.0000

Data Index: 1880401 (The Pile)

Max Activating Token Index: 153

Text #5

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.4966. Min Act: -0.0000

Data Index: 220465 (The Pile)

Max Activating Token Index: 977

Text #6

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.3987. Min Act: -0.0000

Data Index: 385560 (The Pile)

Max Activating Token Index: 212

Text #7

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.2143. Min Act: -0.0000

Data Index: 546325 (The Pile)

Max Activating Token Index: 603

Text #8

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.3262. Min Act: -0.0000

Data Index: 1354259 (The Pile)

Max Activating Token Index: 648

Text #9

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.4215. Min Act: -0.0000

Data Index: 1769647 (The Pile)

Max Activating Token Index: 987

Text #10

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.3051. Min Act: -0.0000

Data Index: 1833046 (The Pile)

Max Activating Token Index: 952

Text #11

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.2046. Min Act: 0.0000

Data Index: 1054851 (The Pile)

Max Activating Token Index: 986

Text #12

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.2821. Min Act: -0.0000

Data Index: 786003 (The Pile)

Max Activating Token Index: 396

Text #13

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.2006. Min Act: -0.0000

Data Index: 227314 (The Pile)

Max Activating Token Index: 189

Text #14

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.1212. Min Act: -0.0000

Data Index: 89264 (The Pile)

Max Activating Token Index: 718

Text #15

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.1260. Min Act: -0.0000

Data Index: 1181661 (The Pile)

Max Activating Token Index: 132

Text #16

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.1187. Min Act: -0.0000

Data Index: 1823208 (The Pile)

Max Activating Token Index: 614

Text #17

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.1693. Min Act: 0.0001

Data Index: 590745 (The Pile)

Max Activating Token Index: 156

Text #18

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.1619. Min Act: -0.0000

Data Index: 1574731 (The Pile)

Max Activating Token Index: 719

Text #19

Max Range: 1.8609. Min Range: -1.8609

Max Act: 1.1352. Min Act: -0.0000

Data Index: 471089 (The Pile)

Max Activating Token Index: 902