Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2774 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.3197. Min Act: -0.0000

Data Index: 1240702 (The Pile)

Max Activating Token Index: 506

Text #1

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.2855. Min Act: -0.0000

Data Index: 1316385 (The Pile)

Max Activating Token Index: 962

Text #2

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.2659. Min Act: -0.0000

Data Index: 1410003 (The Pile)

Max Activating Token Index: 811

Text #3

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.2539. Min Act: -0.0000

Data Index: 1981552 (The Pile)

Max Activating Token Index: 846

Text #4

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.2334. Min Act: -0.0000

Data Index: 1996799 (The Pile)

Max Activating Token Index: 432

Text #5

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.2309. Min Act: -0.0000

Data Index: 1401667 (The Pile)

Max Activating Token Index: 674

Text #6

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.2243. Min Act: -0.0000

Data Index: 499991 (The Pile)

Max Activating Token Index: 589

Text #7

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.2314. Min Act: -0.0000

Data Index: 1570888 (The Pile)

Max Activating Token Index: 784

Text #8

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.2134. Min Act: -0.0000

Data Index: 1414150 (The Pile)

Max Activating Token Index: 442

Text #9

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.2036. Min Act: -0.0000

Data Index: 1464299 (The Pile)

Max Activating Token Index: 549

Text #10

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.2060. Min Act: -0.0000

Data Index: 561336 (The Pile)

Max Activating Token Index: 1000

Text #11

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.2195. Min Act: -0.0000

Data Index: 666076 (The Pile)

Max Activating Token Index: 201

Text #12

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.2085. Min Act: -0.0000

Data Index: 1946617 (The Pile)

Max Activating Token Index: 698

Text #13

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.2122. Min Act: -0.0000

Data Index: 1442343 (The Pile)

Max Activating Token Index: 807

Text #14

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.2118. Min Act: -0.0000

Data Index: 623152 (The Pile)

Max Activating Token Index: 611

Text #15

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.2108. Min Act: -0.0000

Data Index: 665849 (The Pile)

Max Activating Token Index: 970

Text #16

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.1937. Min Act: -0.0000

Data Index: 1742261 (The Pile)

Max Activating Token Index: 116

Text #17

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.1949. Min Act: -0.0000

Data Index: 1273509 (The Pile)

Max Activating Token Index: 326

Text #18

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.1842. Min Act: -0.0000

Data Index: 179846 (The Pile)

Max Activating Token Index: 631

Text #19

Max Range: 0.3197. Min Range: -0.3197

Max Act: 0.1927. Min Act: -0.0000

Data Index: 713905 (The Pile)

Max Activating Token Index: 1016