Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3356 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.2144. Min Act: -0.0000

Data Index: 1767598 (The Pile)

Max Activating Token Index: 346

Text #1

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.1302. Min Act: -0.0000

Data Index: 247156 (The Pile)

Max Activating Token Index: 192

Text #2

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.1372. Min Act: -0.0000

Data Index: 1541609 (The Pile)

Max Activating Token Index: 818

Text #3

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.1267. Min Act: -0.0000

Data Index: 1730648 (The Pile)

Max Activating Token Index: 833

Text #4

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.1256. Min Act: -0.0000

Data Index: 1726601 (The Pile)

Max Activating Token Index: 561

Text #5

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.1135. Min Act: -0.0000

Data Index: 215972 (The Pile)

Max Activating Token Index: 106

Text #6

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.1136. Min Act: -0.0000

Data Index: 1884973 (The Pile)

Max Activating Token Index: 443

Text #7

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.1155. Min Act: -0.0000

Data Index: 1952263 (The Pile)

Max Activating Token Index: 304

Text #8

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.1106. Min Act: -0.0000

Data Index: 898489 (The Pile)

Max Activating Token Index: 88

Text #9

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.1064. Min Act: -0.0000

Data Index: 1590387 (The Pile)

Max Activating Token Index: 217

Text #10

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.1092. Min Act: -0.0000

Data Index: 1622681 (The Pile)

Max Activating Token Index: 203

Text #11

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.1095. Min Act: -0.0000

Data Index: 187535 (The Pile)

Max Activating Token Index: 881

Text #12

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.1127. Min Act: -0.0000

Data Index: 1285137 (The Pile)

Max Activating Token Index: 28

Text #13

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.1084. Min Act: -0.0000

Data Index: 1102654 (The Pile)

Max Activating Token Index: 592

Text #14

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.1053. Min Act: -0.0000

Data Index: 1464795 (The Pile)

Max Activating Token Index: 989

Text #15

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.1119. Min Act: -0.0000

Data Index: 1120786 (The Pile)

Max Activating Token Index: 506

Text #16

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.1092. Min Act: -0.0000

Data Index: 10142 (The Pile)

Max Activating Token Index: 934

Text #17

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.1022. Min Act: -0.0000

Data Index: 1763335 (The Pile)

Max Activating Token Index: 206

Text #18

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.1088. Min Act: -0.0000

Data Index: 763557 (The Pile)

Max Activating Token Index: 809

Text #19

Max Range: 0.2144. Min Range: -0.2144

Max Act: 0.0946. Min Act: -0.0000

Data Index: 937793 (The Pile)

Max Activating Token Index: 495