Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4116 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9303. Min Range: -1.9303

Max Act: 1.9303. Min Act: -0.0000

Data Index: 688832 (The Pile)

Max Activating Token Index: 572

Text #1

Max Range: 1.9303. Min Range: -1.9303

Max Act: 1.1657. Min Act: -0.0000

Data Index: 1024979 (The Pile)

Max Activating Token Index: 795

Text #2

Max Range: 1.9303. Min Range: -1.9303

Max Act: 1.0131. Min Act: -0.0000

Data Index: 1315943 (The Pile)

Max Activating Token Index: 818

Text #3

Max Range: 1.9303. Min Range: -1.9303

Max Act: 1.2329. Min Act: -0.0000

Data Index: 734769 (The Pile)

Max Activating Token Index: 889

Text #4

Max Range: 1.9303. Min Range: -1.9303

Max Act: 1.0261. Min Act: -0.0000

Data Index: 1601135 (The Pile)

Max Activating Token Index: 351

Text #5

Max Range: 1.9303. Min Range: -1.9303

Max Act: 1.0371. Min Act: -0.0000

Data Index: 1344158 (The Pile)

Max Activating Token Index: 707

Text #6

Max Range: 1.9303. Min Range: -1.9303

Max Act: 0.9596. Min Act: -0.0000

Data Index: 517675 (The Pile)

Max Activating Token Index: 853

Text #7

Max Range: 1.9303. Min Range: -1.9303

Max Act: 1.0450. Min Act: -0.0000

Data Index: 160185 (The Pile)

Max Activating Token Index: 1014

Text #8

Max Range: 1.9303. Min Range: -1.9303

Max Act: 1.0968. Min Act: -0.0000

Data Index: 736365 (The Pile)

Max Activating Token Index: 511

Text #9

Max Range: 1.9303. Min Range: -1.9303

Max Act: 1.0753. Min Act: -0.0000

Data Index: 1756709 (The Pile)

Max Activating Token Index: 375

Text #10

Max Range: 1.9303. Min Range: -1.9303

Max Act: 0.9952. Min Act: -0.0000

Data Index: 651346 (The Pile)

Max Activating Token Index: 348

Text #11

Max Range: 1.9303. Min Range: -1.9303

Max Act: 1.0694. Min Act: -0.0000

Data Index: 1085285 (The Pile)

Max Activating Token Index: 352

Text #12

Max Range: 1.9303. Min Range: -1.9303

Max Act: 1.0373. Min Act: -0.0000

Data Index: 113175 (The Pile)

Max Activating Token Index: 309

Text #13

Max Range: 1.9303. Min Range: -1.9303

Max Act: 1.0307. Min Act: -0.0000

Data Index: 1062041 (The Pile)

Max Activating Token Index: 270

Text #14

Max Range: 1.9303. Min Range: -1.9303

Max Act: 0.9675. Min Act: -0.0000

Data Index: 1616032 (The Pile)

Max Activating Token Index: 658

Text #15

Max Range: 1.9303. Min Range: -1.9303

Max Act: 0.9909. Min Act: -0.0000

Data Index: 1866544 (The Pile)

Max Activating Token Index: 380

Text #16

Max Range: 1.9303. Min Range: -1.9303

Max Act: 0.9783. Min Act: -0.0000

Data Index: 1904857 (The Pile)

Max Activating Token Index: 37

Text #17

Max Range: 1.9303. Min Range: -1.9303

Max Act: 0.9209. Min Act: -0.0000

Data Index: 1727118 (The Pile)

Max Activating Token Index: 152

Text #18

Max Range: 1.9303. Min Range: -1.9303

Max Act: 0.9235. Min Act: -0.0000

Data Index: 1063109 (The Pile)

Max Activating Token Index: 401

Text #19

Max Range: 1.9303. Min Range: -1.9303

Max Act: 1.0344. Min Act: -0.0000

Data Index: 1632563 (The Pile)

Max Activating Token Index: 225