Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 880 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.2828. Min Range: -3.2828

Max Act: 3.2828. Min Act: -0.0000

Data Index: 1530682 (The Pile)

Max Activating Token Index: 640

Text #1

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.7660. Min Act: -0.0000

Data Index: 1640966 (The Pile)

Max Activating Token Index: 639

Text #2

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.7881. Min Act: -0.0000

Data Index: 1965216 (The Pile)

Max Activating Token Index: 954

Text #3

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.6277. Min Act: -0.0000

Data Index: 781667 (The Pile)

Max Activating Token Index: 513

Text #4

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.7085. Min Act: -0.0000

Data Index: 98078 (The Pile)

Max Activating Token Index: 420

Text #5

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.6888. Min Act: -0.0000

Data Index: 1720338 (The Pile)

Max Activating Token Index: 462

Text #6

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.4166. Min Act: -0.0000

Data Index: 1475099 (The Pile)

Max Activating Token Index: 905

Text #7

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.5584. Min Act: -0.0000

Data Index: 922203 (The Pile)

Max Activating Token Index: 593

Text #8

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.6168. Min Act: -0.0000

Data Index: 1225921 (The Pile)

Max Activating Token Index: 520

Text #9

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.7280. Min Act: -0.0000

Data Index: 1157212 (The Pile)

Max Activating Token Index: 919

Text #10

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.5881. Min Act: -0.0000

Data Index: 1263902 (The Pile)

Max Activating Token Index: 590

Text #11

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.7193. Min Act: -0.0000

Data Index: 73044 (The Pile)

Max Activating Token Index: 213

Text #12

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.4335. Min Act: -0.0000

Data Index: 765470 (The Pile)

Max Activating Token Index: 205

Text #13

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.3543. Min Act: -0.0000

Data Index: 1805411 (The Pile)

Max Activating Token Index: 16

Text #14

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.4541. Min Act: -0.0000

Data Index: 576525 (The Pile)

Max Activating Token Index: 705

Text #15

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.4617. Min Act: -0.0000

Data Index: 1678261 (The Pile)

Max Activating Token Index: 639

Text #16

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.5483. Min Act: -0.0000

Data Index: 1855211 (The Pile)

Max Activating Token Index: 635

Text #17

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.3154. Min Act: -0.0000

Data Index: 1966884 (The Pile)

Max Activating Token Index: 624

Text #18

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.4235. Min Act: -0.0000

Data Index: 1229827 (The Pile)

Max Activating Token Index: 742

Text #19

Max Range: 3.2828. Min Range: -3.2828

Max Act: 2.4195. Min Act: -0.0000

Data Index: 1138857 (The Pile)

Max Activating Token Index: 425