Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1698 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.2101. Min Act: -0.0000

Data Index: 968972 (The Pile)

Max Activating Token Index: 314

Text #1

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.1773. Min Act: -0.0000

Data Index: 1545827 (The Pile)

Max Activating Token Index: 356

Text #2

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.1422. Min Act: -0.0000

Data Index: 1036168 (The Pile)

Max Activating Token Index: 37

Text #3

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.1503. Min Act: -0.0000

Data Index: 467973 (The Pile)

Max Activating Token Index: 122

Text #4

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.1600. Min Act: -0.0000

Data Index: 839385 (The Pile)

Max Activating Token Index: 43

Text #5

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.1318. Min Act: -0.0000

Data Index: 384717 (The Pile)

Max Activating Token Index: 70

Text #6

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.1187. Min Act: -0.0000

Data Index: 519722 (The Pile)

Max Activating Token Index: 79

Text #7

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.1423. Min Act: -0.0000

Data Index: 107900 (The Pile)

Max Activating Token Index: 21

Text #8

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.1124. Min Act: -0.0000

Data Index: 455281 (The Pile)

Max Activating Token Index: 192

Text #9

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.1317. Min Act: -0.0000

Data Index: 1834380 (The Pile)

Max Activating Token Index: 110

Text #10

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.1181. Min Act: -0.0000

Data Index: 42533 (The Pile)

Max Activating Token Index: 374

Text #11

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.1444. Min Act: -0.0000

Data Index: 217801 (The Pile)

Max Activating Token Index: 57

Text #12

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.0985. Min Act: -0.0000

Data Index: 1751588 (The Pile)

Max Activating Token Index: 60

Text #13

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.1043. Min Act: -0.0000

Data Index: 1936037 (The Pile)

Max Activating Token Index: 69

Text #14

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.1311. Min Act: -0.0000

Data Index: 1539682 (The Pile)

Max Activating Token Index: 85

Text #15

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.1064. Min Act: -0.0000

Data Index: 1458384 (The Pile)

Max Activating Token Index: 59

Text #16

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.1198. Min Act: -0.0000

Data Index: 576745 (The Pile)

Max Activating Token Index: 125

Text #17

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.0838. Min Act: -0.0000

Data Index: 1340626 (The Pile)

Max Activating Token Index: 187

Text #18

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.0910. Min Act: -0.0000

Data Index: 695927 (The Pile)

Max Activating Token Index: 54

Text #19

Max Range: 0.2101. Min Range: -0.2101

Max Act: 0.0987. Min Act: -0.0000

Data Index: 1781377 (The Pile)

Max Activating Token Index: 11